Advanced ASCII Options

The options control how newlines in quoted strings are handled.   To understand the options, it is first necessary to understand how properly formed ASCII, delimited files are  written.  First, newlines (except, obviously, those at the actual end of a line of data)  should always be embedded  in strings that are enclosed in double quotes.  Second, if a double quote in a string, that quote should be doubled.   For example, with \n representing a newline, the following lines are properly formed.

"line 1 \n  line 2", "I said ""hello"""

Stat/Transfer will convert the embedded newline in the first string to a blank (by default), and the second string will become  -

I said "hello".

Delimited ASCII files that are properly formed, such as those written by Excel , will always be read without problems.  However, some software (and people), do not follow the rules.

Supposed that the second of these strings did not have escaped or doubled quotes.  Stat/Transfer would terminate the string at the first embedded quote and then continue looking for the next delimiter.  In the worst case it would chew up multiple lines until it found a matching quote.

The purpose of these options is to help you locate the offending line(s) in a malformed file so that you can do something about correcting them.

Newlines In Quoted Strings

The default is to “process” these.  Newlines (by default) will be converted to spaces.  If you select “warn” or stop, you will get a message that tells you where in your file a newline was detected.  This is obviously most useful if you are not expecting embedded newlines and the problem is with quotes in your strings that are not doubled (this is a surprisingly common problem).

Newline limit per string

By default, this is “none” and, if your file is properly formed, all embedded newlines will be converted and you will have no problems.  However, if there are stray quotes in your file, you should set this to zero. Then, Stat/Transfer will not chew up any additional lines and tell you the first line on which a problem was encountered.

Substitution Character

By default, newlines are converted to blanks.  This will work for any destination format.  However, you may wish to choose a different way of handling them.  You can chose to convert newlines to spaces, vertical bars, colons, or not convert them at all.  If you chose “none”  and  leave them in the output, remember that newlines will cause havoc in such formats as fixed width ASCII.

Force quoted fields to strings

If this is checked, columns that contain any quoted fields will be read as string variables.  This should normally be left checked, however some software will write all fields, including numeric ones with quotes around the values.  If this is the case, uncheck this option.

 

Blank rows:

You can choose to keep blank rows, skip them, or stop the conversion.