Bug 152541

Define and implement Indexing part of the Parsing and Indexing API. This first chunk of work should implement the
following requirements (as they were described on http://wiki.netbeans.org/ParsingAPIRequirements):

I.1: The first time you open the IDE on a new project, the indexing system should scan through all the source files and
index them

I.2: On subsequent startups, only files that have changed since the last IDE session should be indexed again.

I.3: The implementation should handle "index cleanup", e.g. removing data associated with files that are deleted or
renamed. Similarly, at startup, if the IDE notices that files have been deleted (outside of the IDE), the data
associated with these files should be removed as well.

I.4: The indexing system should allow multiple indexers to process a given file. For example, an RHTML/ERB file should
be scanned not just by the Ruby indexer for the Ruby code, but by the JavaScript indexer for potential JavaScript inside
<script> blocks, and possibly by the CSS and HTML indexers (if any) as well. As part of this, indexing must support
indexing of embedded files as well.

I.5: Indexer should be able to skip files they deem unimportant. For example, the JavaScript indexer might want to skip
generated GWT JavaScript files, or optimized (compressed) JavaScript files.

I.9: The indexing system should support isolation, such that each language has its own unique Lucene repository tree it
can use without interference from other languages.

I.10: The indexing system should support versioning, such that
  A. The language plugin can rev its version number whenever it changes the index data incompatibly, and the indexing
system will force reindexing
  B. The indexing infrastructure can rev its own version number whenever it changes the index data incompatibly (for
example, by changing to a new version of Lucene) and the indexing system will force reindexing.

I.11: The indexing system should be tied into the parsing infrastructure in such a way that any attempt to query the
index when the file has been changed, will immediately parse and then run indexing on the file first such that the query
will process up to date indexing data.

I.13: The query system should let you query by exact name, prefix, regular expression, or camel case.

I.15: The Java module has some pretty advanced indexing needs with its .sig files, dependency tracking etc. It must be
possible for the Java module to use the Parsing API and provide its own indexing implementation independent of the
Indexing API. This is probably obvious, but I'm including it for completeness since with this single exception, I would
like the Parsing API and Indexing API to be joined at the hip: "The Parsing and Indexing API".