More Intelligent Searching – Part 2

• This is a continuation of the topic https://jeremybranham.wordpress.com/2010/10/23/more-intelligent-searching-part-1/

One of the next major milestones of searching will be a ‘natural language conversational search engine’.

NL conversational searching will be much different than conventional searching. Today, search engines do not hold context or do any real ‘learning’ from user input.

NL conversational search engines will

• Parse natural language for meaning
• Possibly infer context
• Ask clarifying questions to reduce the number of potential ‘hits’

Think of this style of searching as ‘asking a very smart person who has perfect recall of crawled content’.

More Intelligent Searching – Part 1

Image via Wikipedia

There are 2 major milestones that I am helping work towards in the evolution of on-line searching in the next few years.

1. Semantic indexing/searching – Returning results based on semantic similarity.
2. Natural language conversational searching – The search engine will ‘converse’ with it’s user.

Semantic Indexing

• A web crawler/indexer /spider will parse each page to extract textual content.
• Textual content will be parsed to find ‘meaning’.
• Sentences can be tagged with with semantics symbols, maybe creating a relation between between verb/noun phrases and implied meanings (There is still more work to be done here).
• Search input, using natural language expressions/phrases can be parsed and tagged with semantic symbols.
• The semantic symbols generated from the search input can be compared to indexed symbols.
• Indexed resources containing high semantic similarity can be returned to the user.

There are some great open source projects existing today that can provide the necessary framework to create this first step; most of which are part of the OpenCog framework.