Able Agent Editor Tutorial

This tutorial explains how to use the Able Agent Editor, how to open an example provided with ABLE, and how to create agents to approve mortgages using either neural network or rule-based application techniques.

Starting the agent editor

First, start the Able Agent Editor with the appropriate method for your environment. In Windows or OS/2, go to the /able/bin directory and type

       runnit

On UNIX platforms, go to the /able/bin directory and type

       runnit.sh

After the Able Agent Editor main window appears, you may choose to read through the entire tutorial or select just one lesson at a time from the contents frame.

Using the Able Agent Editor

In this lesson, you are shown how to create, change, connect, inspect, and move the beans that make up an agent. You may wish to review the component descriptions of the Able Agent Editor before proceeding.

Constructing a new agent

To construct a new agent, go to the File pulldown menu and select New...   A cascade menu will appear with a list of available AbleAgents.   Select the type of agent you want to instantiate and one will be created and set as the base container agent for further editing and customization.

Adding beans to an agent

To add an Able bean component to an agent, turn to the desired page on the icon palette and then click on the Able bean icon. The Able beans are divided into several catagories, including Data, Learning, Rules, Agents, and Misc.

To identify each component, just place and pause the cursor over the icon. Popup help text will show the underlying name of the Able bean. When you click on the icon, an instance of the Able bean will be created and added to the Able Agent. The icon will appear in the Agent canvas. At this point you can configure the Able bean, or create buffer, property, or event connections to the Agent or other Able beans in the agent.

Editing an agent's properties

To change the state or configuration of an agent you can select an action from the context or popup menu by right clicking over the agent icon or white space if the agent is the container agent, or you can use the agent customizer by choosing the Properties... item from the context menu.

Initializing a bean

You can change the state or configuration of a AbleBean that is contained in an AbleAgent by clicking the right mouse button over the AbleBean icon in the canvas pane. This will bring up the context or popup menu. To open the customizer, choose the Properties... item from the context menu. While in the Properties customizer, some beans provide buttons to initialize, generate, or otherwise configure themselves which can change the bean's state. A bean that is not initialized will appeared to be greyed-out or disabled on the editor canvas.

Defining sensors and effectors

AbleAgents manage a list of named methods called sensors and effectors.   They provide an easy way for AbleBeans to invoke methods on adaptor beans and to access external application data and actions. To define a Sensor or Effector, open the context menu on the Agent (right click on white space in the Canvas pane) and select Sensors... or Effectors...

A dialog box will appear with a list of the currently defined Sensors and Effectors in a list box.   You can edit existing definitions, or add and remove them from the list.

Saving an agent

Set the agent name by right clicking with the mouse anyplace on the canvas, and selecting Properties... from the context menu. Enter the name, and press OK. Save an agent by selecting Save or Save as from the File pull-down menu. All beans contained in the agent as well as any open inspectors will be written to the serialized file with the name property provided.

Creating beans

There are several methods available for creating AbleBeans. To create a single bean such as an Import bean place the pointer on Figure import32 not displayed. and click the left mouse button. When you release the mouse button, the bean is created and placed on the canvas.

Another method is to copy an existing bean. First, select a bean already on the canvas by placing the pointer over it and clicking the left mouse button. Next, select the Copy choice from the Edit pull-down menu, and then select Paste from the Edit pull-down menu. Copy and Paste are also available from the context menu.

Using context menus

Each bean has an associated pop-up or context menu that is available by clicking the right mouse button on any of the object icons.

Moving beans

To move a bean, position the pointer on the desired object. Press the left mouse button, and hold it down while you move the mouse to drag the object to its new location in the Able Editor canvas. Release the button when the object is in its desired new location.

Connecting beans

There are three types of connections supported in Able: buffer or data connections, property connections, and event connections. Each connect in the same manner. The view determines which connection is visually displayed.

To connect two AbleBeans, right click on the source bean to bring up the context menu, move the cursor over the Connect... item (a cascade menu with appear) and select the desired type of connection. A rubberband line will appear from the source AbleBean. Move the mouse over the desired target or destination AbleBean and click the left mouse button again. A connection will be drawn between the two beans. If it is not visible, change the View selection to the type of connection created.

Removing bean connections

To remove connections from a bean, open its customizer by selecting Properties... from the context menu. Select the Connections tab which displays a list of each of the connection types. Select the connections to be removed and press the Remove button.

Editing bean properties

After an AbleBean is created, there may be properties needed to complete its definition. The attributes themselves depend on the type of bean. For example, an Import bean needs an attribute specified to locate and define the format of the data to be imported. Move the mouse to the bean. Click the right mouse button to show the pop-up or context menu. Select Properties... to display an appropriate bean-specific customizer. Change properties as desired. Click on the OK push button to return to the workplace.

Setting user preferences

The Able Editor allow you to customize settings so that you can change defaults to suit yourself. You do this with the Preferences dialog. To change or view your current choices, select File from the menu bar, and then Preferences... from the pull-down menu.

Starting a Demonstration

To see an example of a simple neural network application, load the animal.ser recognition demonstration from the examples/neural directory.

  1. Select Open agent... from the File pull-down menu.
  2. Use the browse button to navigate to the examples/neural directory, and select the animal.ser. Press the Open button.

After the agent has loaded, notice that the windows are prearranged on the screen. This is a feature of the Able Editor that allows you to end and save a session exactly the way you left it. When you start the Able Editor again, you can pick up where you left off.

Creating an agent

The animal recognition demonstration is an example of a saved agent. To create an agent using the NeuralClassifier-provided beans, follow these steps:

  1. Select the Agents palette.
  2. Click on the NNClassifier agent, and an icon representing the agent will be displayed on the canvas.
  3. Right-click on the icon, and select Properties....
  4. Go to the Neural Classification tab.
  5. Press Browse to choose a training definition file - animal.dfn is in the examples/neural directory.
  6. Generate the beans and connections by pressing the Generate button.
  7. Press the Help button and select Use for more information on training.
  8. Click on the folder to expand the agent in the tree view.
  9. Right click on the bean labelled Import, and go to the Data tab. On this tab you can actually see each data record as it begins to follow the data connections.
  10. Open an inspector on the filter bean labelled GenOutFilter, and accept the default values to display the input and output buffer data.
  11. On the Data tab, press Next record to read the next record from the file. You can change values as you wish. Press the Process button on this panel to feed this record to the network.
  12. Check the inspector for GenOutFilter to see what was calculated.

Inspecting beans

As you have seen in the animal recognition demonstration, the inspectors are a powerful part of the Able Editor. After the beans have been created and configured, you may want to view or inspect what each object does during training. To do this, open an Inspector window on the bean of interest.

Click the right mouse button on the Network object pop-up menu and select the Inspect item. A Text inspector is created, and it displays the Input Buffer by default. Click on the Data item on the menu bar, and select Parameters... from the pull-down menu. Page down until you see the parameter named Max RMS Error. Click on that parameter, and choose a few of your own. When you exit the pull-down menu, the value for each of the parameters you selected will display.

Both text and graphic inspectors are provided. These include variations of bar, line, and scatter plots. Display options are provided for setting titles, legends, axis options, and other graphic options.

The inspector refresh rate can be changed as well. This controls the number of events that the inspector must count before refreshing the screen. Increasing the refresh rate can improve training performance.

If you select View from the pull-down menu, you will see a list of other kinds of inspectors. You can view other types of objects with Inspector windows also. Several Inspector windows can be open at the same time, each on different objects.

A Mortgage Underwriting Application

Suppose you work in the information processing department of a large bank. One line of business in the bank is to issue mortgages to home buyers. Currently, the bank handles about 1000 mortgage applications every month. A mortgage underwriter examines each application and either approves it, rejects it, or requests more information. The bank would like to handle more applications each month but would rather not hire more mortgage underwriters. Your manager has asked you to develop a computer program to pre-evaluate the mortgage applications so the underwriters can make decisions more quickly. Your manager also hints at improving the quality of the underwriting process by decreasing the number of bad debts. This is clearly a classification problem. The underwriter examines the application data and classifies it as either approved or rejected.

This section presents two approaches to solving the mortgage underwriting problem. First, you will use a neural network backward propagation model which is typically used to solve classification problems. Next, you will contrast that approach to one using fuzzy rules.

A Neural Network Approach

This section describes the neural network approach to developing a mortgage underwriting application. These steps are to be discussed:

  1. Defining the source data
  2. Choosing the data representation
  3. Creating a module
  4. Defining acceptable performance
  5. Training the module
  6. Testing the neural network

Defining the Source Data

For auditing purposes, the bank has archived all mortgage applications handled in the past seven years. You decide to use this information to train a neural network to mimic a mortgage underwriter. The database contains the following information about the applicant, the property, and the loan:

Employment data
Monthly income, years employed
Finances
Total savings
Credit rating
Number of good and poor ratings
Property
Appraised value, location, structure value
Loan
Amount, PITI 1 (monthly payment)
Underwriter response
Approval or rejection reason codes

You could try to use this data as it is to train the neural network. The desired output of the neural network is the underwriter's response and the input data is the rest of the application information. However, it is useful to understand how the data is used to make decisions so you can explicitly present the most important information to the neural network. Otherwise the neural network takes a long time to train, and it may make decisions based on irrelevant features in the data.

According to a mortgage underwriter, every application is different and many factors influence the decision. As "rules of thumb", though, an application is usually rejected in the following cases:

In these rules, the underwriter is making use of some ratios that are calculated from the raw data. To present this information explicitly to the neural network, create a logical view of the database to add these ratios:

Since the relevant information about credit rating is coded in the CTS field, you decide to omit the actual number of good and poor ratings from the database view. You have now defined the source data.

The next step is to subset the data into two or more files that can be used for training and testing the neural network. Too many records in the training file can make training cumbersome; but if too few are used, they may not be a representative sample.

The records in the testing file are used as a "control group". You use the test records to determine how well the neural network performs on data it has not seen before. This is an indication of how well the neural network has generalized from the training records.

You decide to take a random sample from the bank's database to create a training file and testing file with 20 records each. In a real application, of course, it would be better to have larger files than this. The files could be database or spreadsheet, but this tutorial assumes the format is space-delimited text.

Each of these files has the following fields:

Figure 5. Source Data Definition

Database Field  Description 
Income  Whole number up to $10 000 
YearsEmployed  Whole number up to 30 
Savings  Whole number up to $10 000 
AppraisedValue  Whole number up to $1 000 000 
Location  Character, one of RURAL, SUBURB, URBAN 
StructureValue  Whole number up to $1 000 000 
LoanAmount  Whole number up to $1 000 000 
PITI  Whole number up to $10 000 
LTV  Real number 0.00 to 1.00 
PTI  Real number 0.00 to 1.00 
STV  Real number 0.00 to 1.00 
CTS  Real number 0.00 to 1.00 
Approved  Character, either YES or NO 
Reason1  Character, either YES or NO 
Reason2  Character, either YES or NO 
Reason3 Character, either YES or NO 
Reason4  Character, either YES or NO 
Reason5  Character, either YES or NO 

The last six fields represent the underwriter's response. If the loan is approved, all the reason fields have the value NO. If the loan is not approved, one reason field will have the value YES. The five reasons given for rejecting the loan application are:

  1. insufficient income
  2. unstable employment
  3. poor credit
  4. low property value
  5. low structure value.

The record format for the data file is provided in a Definition file in the examples/neural/MortgageTrain.dfn file. You could use your favorite editor to create a definition file or use the one we provide. If you create your own, the field labelled Approved above is the class field. This field represents the mortgage underwriter's decision and is the field that the network predicts based on the other data values.

Creating an NNClassiferAgent

If you have not already done so, start the Able Editor interactive development environment. The following sections show how to create an Able Neural Classifier agent consisting of five objects: an Import bean for training data, an Import bean for testing data, a Filter bean for input, a neural network bean, and a Filter bean for output.

Note: In this tutorial, fields that are not discussed should not be changed from their initial values.

  1. Create the agent.
    Select the Agents palette. Click on the NNClassifier agent, and an icon representing the agent will be displayed on the canvas.
  2. Configure the agent.
    Right-click on the icon, and select Properties...; Go to the Neural Classification tab. Press Browse to choose a training definition file - MortgageTrain.dfn is in the examples/neural directory. Generate the beans and connections by pressing the Generate button.
  3. Set the network architecture.
    Leave the network bean's architecture with no hidden layers.
  4. Create an inspector to monitor network accuracy during training.
  5. Click on the folder to expand the agent in the tree view.
  6. Click on the bean labelled Network in the tree, and then right-click on the icon labelled Network in the canvas to open a new Inspector.
  7. Select the Data pull-down menu, and choose Parameters...
  8. Click on the Remove all button on the lower right.
  9. Select aveRMSerror, and the press the Add button.
  10. Change Number of Data points to 200.
  11. Press OK.
  12. On the inspector panel, select View and switch from text to time plot.
  13. From the Options pulldown, select Properties. Select the Y axis tab, and set the axis origin to 0 and the maximum to 1. The RMS error will always be between these values.
  14. Create an inspector to monitor number of passes as above.
    You can also tell how many passes were performed by the number of data points.
  15. Train the network.
    Return to the Properties panel and neural classifier tab for the agent. Set the Minimum Percent Correct to 90, and Maximum Percent Incorrect to 5, and Maximum passes to 100. Press the Start button.
  16. Watch the inspector
    As the network trains, you should see the average RMS error begin close to 1.0, and lower with time. Press the Help button and select Use for more information on training.
  17. Still training?
    Return to the Properties panel and neural classifier tab for the agent. Set the Minimum Percent Correct to 60, and Maximum Percent Incorrect to 40. To discard what the network has learned so far, you could press Reset Beans to reset the network weights. To save time, just press the Start button once again. Be patient, watch the error rate come down, and it should bottom out at about 0.08.
  18. Experiment with the network architecture
    Change the number of units in the first hidden layer from 0 to 10. Press Generate Beans to update the network bean's architecture, and press the Start button. With the current implementation, Generating Beans will remove the Inspectors. Does it train faster this time?

You've now created a neural classifier agent, trained it, and are ready to validate that it works with independent data.

Validating the Network

You need to establish a validation criteria for the accuracy of the neural network on the test data set file. These are records the neural network has not seen during training. You decide to accept the neural network if it can produce the desired response for at least 90% of the test records.

Switch to the test data set. One is provided called MortgageTest in examples/neural/MortgageTest.dat. Set the agent mode to Test. Create text inspectors to monitor the average RMS error and bad pattern ratio parameters. Press Start. Check to see that the values are within the limits you set. Note: the test data is as real as the training data.

You have just completed testing the neural network using the testing data set file. The test data set file contains records that were not used in the training processes. How well the neural network performs on this data is a good indicator of how it performs when it is deployed as an application.

A Fuzzy Rule-Based Approach

Sorry, but this tutorial is not yet available. See the ABLE Project Page for its availability.