Javafamily Release Notes

# Release Notes

October 6, 2003
The javafamily plug-in is a collection of prototypes that illustrate how the existing (mostly Java) infrastructure can be extended to support Java-like languages like JSP. Since this is a first cut, the project still contains a number of workarounds and duplications. However, it is our ultimate goal to eliminate all of these in order to make supporting Java-like languages smooth and simple.
This initial release contains:

- a JSP editor with reconciling,
- a JSP aware Java search,
- a JSP aware Rename Type refactoring that uses the new Refactoring Participants.
- a Tomcat launcher and JSP source locator

In addition this document contains a collection of findings we made while developing the various pieces.

## Known issues and limitations:

- Currently the JSP index isn't persisted and automatic indexing of JSP files is not enabled. So if you want to try the Rename Type refactoring or the JSP/Java search, you'll need to turn on JSP indexing via Window > Preferences > Java > JavaFamily Example > Start JSP indexing. Without this a JSP/Java search or a Rename Type refactoring will not find Java types in non-Java resources.
- JSP Search works only for types (because JSP indexing is only done for types).

## Reconciling Steps

Findings while developing the JSP Editor:

- JSR-045 specifies a line-oriented mapping between several input files (translated-source) and one output file (translated-source or final-source). The output file can either be the final-source or multiple output files (translated-source) can serve as input for a next mapping (e.g. MyCustomFormat -> JSP -> Java). Each step (language-translator) produces a SMAP file (.smap). The mapping cannot be used to exactly map positions inside a line.
- The compiler aborts compilation if it hits some severe errors. Because a JSP line sequence is not necessarily the same as in the intermediate Java file it might happen that the compiler stops very early on an error that corresponds to a JSP line which is at the very end of the file.
- An annotation position reported on the Java file can correspond to

- an annotation in the JSP file with adapted offset and length
- several annotation in the JSP file either distributed in the same Java partition or over several partitions
- no annotation at all, e.g. if the error is caused by the translator or by a wrong Java build path

- A JSP file can include other JSP files. If an SMAP based approach is used and one of the included JSP files is opened in the editor then we need to find the correct SMAP file.
- For most of the tags in the JSP file it is not possible to provide a generic parsing and mapping. This has to be defined for each tag. The generated Java code and the mapping are tightly coupled and should be kept in a single class that we call tag handler. A tag handler might handle more than one tag.
- Special tags are needed to handle non-tagged sections like HTML in a JSP file.
- The default implementation of the tag handler must be extensible in order to allow tag lib providers to extend the handler.
- Even if we decide to use an existing translator we will have to provide tag handlers that can map positions in a translated line back to the position in the corresponding source line.
- Unless we don't build a fully generic editor which can handle several file types (e.g. XML, Java and JSP) there's no need to have global tag handler factory which manages different languages.
- Currently only the source line is passed to the tag handler. If JSP allows that tag (lib) attributes span multiple lines then we must change the interface and pass sourceLineStart and sourceLineEnd.

## Towards Language Neutral Search

We tried to reuse the Java indexing/searching infrastructure from jdt.core for non-Java JSP files. This was easily possible for the indexing side, because that part makes almost no assumptions about was is being indexed. On the other hand, searching was more challenging because the search infrastructure makes hardwired assumptions about the type of files referenced from the index. So if a file doesn't have the .java extensions it is assumed to be a .class file without further checking of its extension. As a consequence it was not possible to get *.jsp files from a search because the search engine would treat them as class files and would run in a NPE.
To solve this issue we created a new package "org.eclipse.core.indexsearch" as a starting point for an indexed search component. The package provides a minimal but sufficient API for indexing/searching non-Java files and uses the existing mechanisms from jdt.core as far as possible.
As a first use case for this indexed search we combined the Type Rename refactoring participants with a JSP search engine based on "org.eclipse.core.indexsearch" and a simple JSP parser (org.eclipse.jsp.AbstractJspParser). With this you can rename Java types and automatically rename all occurrences of the Java type in JSP tags as well.
In a second use case we created a new Java/JSP Search page that combines the existing Java search with the JSP search. As a result you can search for a Java type and can find occurrences not only in Java source but in JSP tags as well. The current implementation copies some of the classes from "org.eclipse.jdt.internal.ui.search" in order to be able to show non-Java files in the Java search result viewer. However, the ultimate goal would be to provide a new extension point for Java search that would allow for plugging in arbitrary "search participants".
Please note: it is not possible to automatically have every Java search return JSP search results as well, because some places in Eclipse assume that the search results only contain Java source, and they would fail miserably when confronted with non-Java files, e.g. JSP files.
Design considerations for an indexed search component:

- language independence
- a single index and a single indexer thread for all languages
- minimal API surface area
- flexible queries
- background indexing and searching
- not tied to IResources

### The API

#### Search Engine: class `SearchEngine`

- API entry point
- combines indexing API with searching API
- manages concurrency issues between indexing access and searching
- indexing based on the JobManager (current impl. delegates to jdt.core IndexManager)
- index requests are added as AddFileToIndex subclasses (like today)
- lifecycle issues: loading/saving index
- no notion of "Scope"; scope can be introduced as an implementation detail of the IIndexQuery

`/**
* Perform the given query against the index and return results via the resultCollector.
*/
public void search(IIndexQuery search, ISearchResultCollector resultCollector,
IProgressMonitor progressMonitor, int waitingPolicy);
`

#### Search query: interface `IIndexQuery`

An IIndexQuery is used to perform a query against the searching framework.

    /**
     * Compute the list of paths which are keying index files and add them to the given list.
     */
    void computePathsKeyingIndexFiles(ArrayList requiredIndexKeys);

    /**
     * Perform the query on the given index and adds the paths of all found documents to the given collector.
     */
    void findIndexMatches(IIndex index, PathCollector collector, IProgressMonitor progressMonitor) throws IOException;

    /**
     * Locate all matches of this query in the given file candidate and return them via the resultcollector.
     */
    void locateMatches(IFile candidate, ISearchResultCollector resultCollector);

#### Search results: interface `ISearchResultCollector`

- search results reported via ISearchResultCollector

`/**
* Accepts the given search result.
*
* @param resource the resource in which the match has been found
* @param start the start position of the match, -1 if it is unknown
* @param length the length of the match
* @exception CoreException if this collector had a problem accepting the search result
*/
public void accept(IResource resource, int start, int length) throws CoreException;
`

### Open Issues

- the PathCollector API has Java-specific methods which are not used
- Transparent search in WorkingCopies/Buffers: whenever a file resource in opened in an Editor, search should transparently search in the buffer and not in the underlying resource. Two options to make buffers available to SearchEngine:

- as argument to search(...) method
- whenever buffer is created or deleted it is registered with the SearchEngine

- No common abstraction for the "content" being indexed:
  In the API from above "content" is represented as an IFile in locateMatches(...) and an IResource in ISearchResultCollector.accept(...). However, to transparently deal with "buffers" there seems to be a need for a more abstract interface for content access.

##
Debugging a JSP

This section describes how to launch a Tomcat server and debug a JSP and associated Java code being developed in a workspace.

### Tomcat and Project Configuration

JSP debugging with the javafamily plug-in requires that you have a local installation of Tomcat, version 5.0.2 or higher. Tomcat can be configured to locate a web application in an arbitrary location. Thus, a web application (JSPs and Java code) can be developed in an Eclipse workspace, and Tomcat can be configured to locate the web application in the associated location in the file system.

A project structure conforming to that of an expanded WAR (Web application ARchive) is required. Thus, you must create a Java project that contains the following directory structure.

- `root-web-app-folder`
  - `WEB-INF`
    - `classes`
    - `lib`

A root container is used to store a `WEB-INF` folder, which contains a `web.xml` file describing the web application. Note that the root container may be a Java project itself. The `classes` folder contains any required class files (i.e. client code that is not part of the standard class libraries, or common libraries shipped with Tomcat). Thus, your Java project must be configured to have an output location pointing to the `classes` folder. Similarly, the `lib` folder contains any required jars. JSPs should be developed in the web application's root folder.

To configure Tomcat to find the web application, a context entry is added to Tomcat's `server.xml` configuration file (found in the `/conf` directory of your Tomcat installation), for each external web application being developed. Following is an example extract of context entries for the default context (identified by the empty `path` attribute), and a sample web application being developed in an Eclipse workspace (in this case, rooted at `d:\testspaces\test-space\JSPs\webapps\myWebApp`).

    <Context path="" docBase="ROOT" debug="0"/>
    <Context path="/myWebApp" docBase="d:\testspaces\test-space\JSPs\webapps\myWebApp" debug="0"/>

The corresponding Java project is "`JSPs`", containing the folder "`webapps`", etc. The project may also contain Java source code (for example, in a "`src`" folder), and the output location is set to "`JSPs\webapps\myWebApp\WEB-INF\classes`". JSPs are created in the "`myWebApp`" folder.

Update: The tomcat 5.0 documentation specifies that "it is NOT recommended to place <Context> elements directly in the server.xml file". Instead, create a new file `myWebApp.xml` in the folder `$CATALINA_HOME/conf/Catalina/localhost` with the following content:

    <Context path="/myWebApp" docBase="d:\testspaces\test-space\JSPs\webapps\myWebApp" debug="0"/>

### Example JSP Project

An example JSP project is included in the `exampleJspProject.zip` file, found in the `stuff` folder of the `javafamily` plug-in. It demonstrates the directory structure and includes a simple JSP and associated Java class.

### Setting a Breakpoint in a JSP

To create a breakpoint in a JSP, double click in the JSP edtior ruler on the line where you want the breakpoint. The breakpoint will appear in the Breakpoints view as well as the editor's vertical ruler. Breakpoints can also be placed in Java source code.

### Launching Tomcat

To debug a JSP, Tomcat must be launched in debug mode. This can be done with a "Tomcat Server" launch configuration.

To create a Tomcat launch configuration for a web application in the workspace, open the launch configuration dialog, and create a new "Tomcat Server" launch configuration. You will notice an error message indicating that the Tomcat install directory or "${catalina_home}" does not exist. To solve this problem, set the value of the `${catalina_home}` string variable in the Run/Debug > String Substitution preference page to the location of your Tomcat installation. For example, "`d:\jakarta-tomcat-5.0.2`".

On the Tomcat tab, also provide the location of your web application - for example "`JSPs\webapps\myWebApp`". This can also be done by pressing the Browse button and selecting the associated web application root folder in the workspace.

Now you can launch Tomcat by pressing the Debug button.

### Summary to Debug a JSP

The following steps must be performed to debug a JSP

- Install Tomcat 5.0.2 or higher
- Set the value of `${catalina_home}` (Run/Debug > String Substitution preference page) to point to the Tomcat installation
- Create a project (web app) in your workspace in the expanded WAR format
- Ensure the output location of the project is the `WEB-INF/classes` folder
- Configure Tomcat to locate the web app by adding a context entry in the `server.xml` file
- Code JSPs in the web app's root folder
- Create a Tomcate Server launch configuration that includes the web app location (located on the "Tomcat" tab of the launch config)