Posted on

Probabilities are Better than Scores

Strategic Analytics slide from Fair Isaac Interact on 2007 mortgage meltdownDuring a panel at Fair Isaac’s Interact conference last week, a banker from Abbey National in the UK suggested that part of the credit crunch was due to the use of the FICO score.  Unlike other panelists, who were former Fair Isaac employees, this gentleman was formerly of Experian!  So there was perhaps some friendly rivalry, but his point was a good one.  He cited an earlier presentation by the founder of Strategic Analytics that touched on the divergence between FICO scores and the probability of default.  The panelist’s key point was that some part of the mortgage crisis could be blamed on credit scores, a point that was first raised in the media last fall.

The FICO score is not a probability. 

Fair Isaac people describe the FICO score as a ranking of creditworthiness.  And banks rely on the FICO score for pricing and qualification for mortgages.  The ratio of the loan to value is also critical, but for any two applicants seeking a loan with the same LTV, the one with the better FICO score is more likely to qualify and receive the better price.

Ideally, a bank’s pricing and qualification criteria would accurately reflect the likelihood of default.  The mortgage crisis demonstrates that their assessment, expressed with the FICO score, was wrong.  Their probabilities were off. Continue reading “Probabilities are Better than Scores”

Posted on

Super Crunchers: predictive analytics is not enough

Ian Ayres, the author of Super Crunchers, gave a keynote at Fair Isaac’s Interact conference in San Francisco this morning.   He made a number of interesting points related to his thesis that intuitive decision making is doomed.   I found his points on random trials much more interesting, however.

In one of his examples on “The End of Intuition”, a computer program using six variables did a better job of predicting Supreme Court decisions than a team of experts.  He focused on the fact that the program “discovered” that one justice would most likely vote against an appeal if it was labeled a liberal decision.    By discovered we mean that a decision tree for this justice’s vote had a top level decision as to whether the decision was liberal, in which case the program had no further concern for any other information.  Continue reading “Super Crunchers: predictive analytics is not enough”

Posted on

A Common Upper Ontology for Advanced Placement tests

I have previously written about the lack of a common upper ontology in the semantic web and commercial software markets (e.g., business rules).  For example, the lack of understanding of time limits the intelligence and ease of use of software in business process management (BPM) and complex event processing (CEP).  The lack of understanding of money limits the intelligence and utility of business rules management systems (BRMS) in financial services and the capital markets.   And, more fundamentally, understanding time and money (among other things, such as location, which includes distance) requires a core understanding of amounts.

The core principle here is that software needs to have a common core of understanding that makes sense to most people and across almost every application.  These are the concepts of Pareto’s 80/20 Principle.  A concept like building could easily be out, but concepts like money and time (and whatever it takes to really understand money and time) are in.  Location, including distance, is in.  Luminousity could be out, but probably not if color is in.  Charge and current could be out, but not if electricity or magnetism is in.  The cutoff is less scientific than practical, but what is in has to be deeply consistent and completely rational (i.e., logically rigorous).[2] Continue reading “A Common Upper Ontology for Advanced Placement tests”

Posted on

Real AI for Games

Dave Mark’s post on Why Not More Simulation in Game AI? and the comments it elicited are right on the money about the correlation between lifespan and intelligence of supposedly intelligent adversaries in first person shooter (FPS) games.  It is extremely refreshing to hear advanced gamers agreeing that more intelligent, longer-lived characters would keep a game more interesting and engaging than current FPS.  This is exactly consistent with my experience with one of my employers who delivers intelligent agents for the military.  The military calls them “computer generated forces” (CGFs).  The idea is that these things need to be smart and human enough to constitute a meaningful adversary for training purposes (i.e., “serious games”).  Our agents fly fixed wing and rotary wing aircraft or animate special operations forces (SOFs) on the ground.  (They even talk – with humans – over the radio.  I love that part.  It makes them seem so human.) Continue reading “Real AI for Games”

Posted on

The Semantic Arms Race: Facebook vs. Google

As I discussed in Over $100m in 12 months backs natural language for the semantic web, Radar Networks’ Twine is one of the more interesting semantic web startups.  Their founder, Nova Spivak, is funded by Vulcan and others to provide “interest-driven [social] networking”.  I’ve been participating in the beta program at modest bandwidth for a while.  Generally, Nova’s statements about where they are and where they are going are fully supported by what I have experienced.  There are obvious weaknesses that they are improving.  Overall, the strategy of gradually bootstrapping functionality and content by controlling the ramp up in users from a clearly alpha stage implementation to what is still not quite beta (in my view) seems perfect. 

Recently, Nova recorded a few minute video in which he makes three short-term predictions: Continue reading “The Semantic Arms Race: Facebook vs. Google”

Posted on

Adaptive Decision Management

hr-dashboard.jpg

In this article I hope you learn the future of predictive analytics in decision management and how tighter integration between rules and learning are being developed that will  adaptively improve diagnostic capabilities, especially in maximizing profitability and detecting adversarial conduct, such as fraud, money laundering and terrorism.

Business Intelligence

Visualizing business performance is obviously important, but improving business performance is even more important.  A good view of operations, such as this nice dashboard[1], helps management see the forest (and, with good drill-down, some interesting trees). 

With good visualization, management can gain insights into how to improve business processes, but if the view does include a focus on outcomes, improvement in operational decision making will be relatively slow in coming.

Whether or not you use business intelligence software to produce your reports or present dashboards, however, you can improve your operational decision management by applying statistics and other predictive analytic techniques to discover hidden correlations between what you know before a decision and what you learn afterwards to improve your decision making over time.  Continue reading “Adaptive Decision Management”

Posted on

Cyc is more than encyclopedic

I had the pleasure of visiting with some fine folks at Cycorp in Austin, Texas recently.  Cycorp is interesting for many reasons, but chiefly because they have expended more effort developing a deeper model of common world knowledge than any other group on the planet.  They are different from current semantic web startups.  Unlike Metaweb‘s Freebase, for example, Cycorp is defining the common sense logic of the world, not just populating databases (which is an unjust simplification of what Freebase is doing, but is proportionally fair when comparing their ontological schemata to Cyc’s knowledge).  Not only does Cyc have the largest and most practical ontology on earth, they have almost incomprehensible numbers of formulas[1] describing the world.   Continue reading “Cyc is more than encyclopedic”

Posted on

Harvesting business rules from the IRS

Does your business have logic that is more or less complicated than filing your taxes?

Most business logic is at least as complicated.  But most business rule metaphors are not up to expressing tax regulations in a simple manner.  Nonetheless, the tax regulations are full of great training material for learning how to analyze and capture business rules.

For example, consider the earned income credit (EIC) for federal income tax purposes in the United States.  This tutorial uses the guide for 2003, which is available here. There is also a cheat sheet that attempts to simplify the matter, available here. (Or click on the pictures.)

eitc-publication-596-fy-2003.jpgeitc-eligibility-checklist-for-tax-year-2003.jpg

What you will see here is typical of what business analysts do to clarify business requirements, policies, and logic.  Nothing here is specific to rule-based programming.  Continue reading “Harvesting business rules from the IRS”

Posted on

Ontology of time in progress – amounts needed

Recent posts on money and time have produced some excellent comments and correspondence.  There is even recent OMG effort that is right on the money, at least concerning time.  For details, see the Date-Time Foundational Vocabulary RFP.  I am particularly impressed with SBVR “Foundation” Vocabularies, which I understand Mark Linehan of IBM presented last week at an OMG meeting in DC[1].

Mark’s suggestions include establishing standard upper ontologies for:

  1. Time & dates
  2. Monetary amount
  3. Location
  4. Unit of measure
  5. Quantities, cardinalities, and ratios
  6. Arithmetic operations

I will skip operations for now since they are not taxonomic concepts but functional relationships involving such concepts.  I believe the post on CEP and BPM covered time in adequate detail and the post on Siebel’s handling of foreign exchange covered the currency exchange aspects of money.  It only touched on the more general concept of amounts that I will focus on here.

The remaining concepts are common to almost every application conceivable.  They are some of the most primitive, domain-independent concepts of a critical and practical upper ontology.  They include: Continue reading “Ontology of time in progress – amounts needed”