Automatic Optimization of Target Types

Stat/Transfer attempts to produce the smallest possible output data set. After reading your data, Stat/Transfer will make an optimization pass during the transfer, to determine more information about each variable and then will assign target output types.


First, before the transfer, information from the data file dictionary is read.  Unfortunately, for some input data types, this information is not sufficient to do anything other than set all of the output variable types to 'float'.


Stat/Transfer will then make an additional optimization pass through your data.  By default this will occur during the transfer.  This pass will only be performed if the file contains any string variables or if the selected output file type is such that the output file could be made smaller by optimization.  Some output file types, such as Stata, have a rich assortment of storage types and benefit from optimization.  Others, such as worksheets or files that have only one numeric type (SPSS for example), do not benefit.  Optimization must be performed for any file with string variables, because the length of output strings can potentially change as they are transcoded to different output encodings.


Stat/Transfer can determine whether any variables can be represented as integers, and, for those cases, it can determine the smallest possible integral type that can be used to represent the data.  Further, if a variable cannot be represented by an integral type, Stat/Transfer can automatically determine whether it can be represented by a float instead of a double without a loss of information.  Information on the maximum length of string variables after conversion to the output encoding is also accumulated, so that truncation will not occur and to enable variables to be be stored in variables of the smallest possible length.


In almost all cases it is appropriate to accept the output types that Stat/Transfer chooses.  However, there are times that you may wish to override these defaults and set the output types yourself.


Optimization, by default, takes place during a transfer. The Optimize button is used to optimize manually before the transfer.  This allows you to see the target types that Stat/Transfer has chosen for your variables before they are transferred, and to change any output types that you wish.  This is discussed in the section Changing the Types of Output Variables.