Illegal Characters

When IBIS-View transforms XML to HTML, it gets tripped-up when it encounters characters that are not strict ASCII text.

For instance, let's say I'm working on a new query, and I'm using some text that I've copied from a Word document, or other documentation that is not strictly text. That copied content may contain non-ASCII characters.

Common culprits are the following:

& – (mdash) “ ” ’ (directional quotes and apostrophes) / < > #

When you go to test the query in our local development environment, and you try to select that query page or run that query, and result looks like the following:

We review the Tomcat Console log, and an error similar to the following is displayed:

The file shown in the error message (in this case _YRBSSelection.htm) contains many, many include files, so it can be difficult to tell where the error may be just by inspecting the text.

We could diff the files with ones that have been checked in, or we could use Notepad++ and use a regular search for all illegal characters in .xml files

Here's an image of Notepad++ Find In Files Search Screen

Note the following:

Find what: is a regular expression:


This could be improved by checking anything below x32 that is not a CR(x13) or LF(x10) or tab(x8), but this will work for now
It will search for any character that is not in the range of ascii characters

For Filters: Limit the file type to .xml

The search directory should be something below ibisph-view-content

Make sure that the Regular Expression Button is checked in the search mode box.

Below is an example of what may cause the error:

It should be noted that illegal characters in xml files are not limited to issues with Queries. The illegal character error can happen anywhere in IBIS. When you save content to the Admin DB, the Admin application checks for illegal characters and converts them to "?" before it saves the content. But for other IBIS pages, such as query pages and static pages, you will need to do the checking yourself.

More information about other basic HTML and XML issues may be found here:

Last modified 21 months ago Last modified on 04/29/17 09:14:47

Attachments (4)

Download all attachments as: .zip