This paper introduces a framework for building knowledge-based information agents. The knowledge-based approach that separates knowledge bases from other processes supports easy agent generation and adaptation. A new agent can be generated by adding new knowledge bases and an agent can be adapted by changing the knowledge bases. The classification of knowledge into three categories enables knowledge reuse and sharing. The general knowledge base is completely reusable and is built as part of the agent shell. The domain knowledge need to be changed for new domains and the site specific knowledge needs to be extended or learned for new Web sites.
This research focuses on semi-structured data and has been successful at extracting information from multiple Web sites in limited domains. With the rapid growth of the Internet, more and more services are become available online. Many of them present semi-structured data, for example, product catalogs, weather forecasts, phone books and stock market quotations. Our system is very useful for building information extraction systems for these online services. Users can generate their own information extraction system by creating knowledge bases and plugging them into our reusable shell. We believe this is much easier and faster than building a system from scratch.
We are currently working on how to learn part of the knowledge automatically. An algorithm is developed to learn site specific information extraction patterns from tabular Web pages [5]. Future work is needed to improve the learning techniques and to reduce manual work required for building and updating the knowledge bases.