# Instructions ## List of features This tool has a feature to convert input CSV files to the one which MANUFACIA can read. Additionally, it can also process input files described as below. | Feature | Description | |:-----------------------------------------|:-------------------------------------------------| | [Show version number](#show-version) | Shows the version number of the tool. | | [Check format](#format-check) | Checks CSV file format and logs the result. | | [Correct format](#correct-format) | Based on the error information, it will automatically correct the format so that MANUFACIA can read.
It only logs where there is an error. | | Process data: [Divide file](#divide-file) | Divides a file into several files with the size of specified line numbers. | | Process data: [Extract rows](#extract-rows) | Extract the block of rows specified. | | Process data: [Extract columns](#extract-columns) | Extract the block of columns specified. | | Process data: [Thin out data](#thin-out-data) | Thin out the data every n-th of the specified number within data area. | | Process data: [Swap rows and columns](#swap-rows-columns) | Swap row and column. | | Process data: [Replace header line](#replace-header-line) | Add/overwrite CSV file header. | The options below are the ones to be used together with the options above, that do not have any effect by itself. | Additional feature | Description　 | |:----------------------------------------|:-----------------------------------------------------| | [Input data path](#input-data-path) | Specifies file path where the input data files are. | | [Rebuild time data](#rebuild-time-data) | Rebuild time data while processing data. | | [Force data processing](#force-process) | Ignores errors and force process data files. | ```{important} 1. Correct format first and then process data, because processing data will be only done for CSV files in proper format. 2. Log file or processed CSV files will be written in the same directory where the original CSV files are. 3. To process data, only one of the features above can be done. If there are multiple options specified in the command line, only the first option will be taken for the data processing. ``` ## Feature detail (show-version)= ### Show version information of the tool ``` csv_checker -i ``` Displays the following information about this tool. - Version number - Default setting values (format-check)= ### Format check ``` csv_checker ``` Verifies CSV files in the current directory. To check CSV files in arbitrary disk location, use [```-p```option](#input-data-path). The result will be logged in the following style. ```{list-table} Format check result :header-rows: 1 :widths: 1 5 * - Result - Description * - OK (correct) - ・Header comments are only in the first line of the file.
・Values are separated by comma (,).
・File encode is UTF-8. * - Error1 - There are comments in multiple lines. * - Error2 - Header comments are not the same in different files. * - Error3 - Delimiter is semicolon (;). * - Error4 - Delimiter is white space or tab. * - Error5 - Data are enclosed by double ("")/single('') quotation marks. * - Error6 - There are white space before/after the value. * - Error7 - Double-byte characters are used for the value. * - Error8 - Non-numeric characters are used. * - Error9 - Value is NULL or white space. * - Error10 - BOM (EF,BB,BF) is added to the file. * - Error11 - File is not encoded in UTF-8. * - Error12 - Slash (/) is used. * - undefined - Other errors; no header line, there is no data lines etc. ``` (correct-format)= ### Correct format ``` csv_checker -c ``` It will correct errors by running format check so that MANUFACIA can read. The errors will be corrected in the following way. ```{list-table} Correction method due to errors :header-rows: 1 :widths: 1 5 * - Error number - Correction method * - Error1 - Takes only the last comment line and removes the rest. * - Error2 - No correction. * - Error3 - Replaces the delimiter with comma (,). (Only for header line.) * - Error4 - Replaces the delimiter with comma (,). (Only for header line.) * - Error5 - Removes double/single quotation marks. * - Error6 - Removes white space. * - Error7 - Converts to one-byte character. * - Error8 - No correction. * - Error9 - Inserts 0. * - Error10 - Removes BOM (three bytes). * - Error11 - Converts to UTF-8 encode. (Only if the input file is in SHIFT-JIS encode.) * - Error12 - Removes slash. * - undefined - No correction. ``` ### Process data The following features will be done by adding corresponding option to call csv format checker. Multiple options at the same time will not be accepted. Both row and column number start from 1. (divide-file)= #### Divide file ``` csv_checker -d ``` Divides the data area of the file so that each file will have at most the number of lines specified with <num> and add header line at the first line. (extract-rows)= #### Extract rows ``` csv_checker -e ``` Extracts the data area block starts from <start> and ends at <end> from the original file and create a new one. Start line number <start> should not be 1 to exclude header line. If end line number <end> is not given, then it will extract until the end of the file. Header line will be copied to the first line of all the files. (extract-columns)= #### Extract columns ``` csv_checker -r ``` Extracts the data area block starts from <start> and ends at <end> from the original file and create a new one. If the specified numbers are above the number of columns available in the CSV file, error will be logged. (thin-out-data)= #### Thin out data ``` csv_checker -t ``` Think out data lines specified by <interval> from the data area top (2nd line) until the end of the file. ```{admonition} Example of thinning out data (-t option) If the interval is three, the lines in the parentheses () will be removed. {header line, 1, 2, (3), 4, 5, (6), 7, 8, (9), ...} ``` (swap-rows-columns)= #### Swap rows and columns ``` csv_checker -s ``` Swaps the rows and columns of the data. Header line will also be included. (replace-header-line)= #### Replace header line ``` csv_checker -h ``` If there is header.txt in the current directory, it will replace header line of existing CSV files with the string specified in header.txt. The number of columns in the header.txt should be equal to the one of data area. Only the first line will be used if there are multiple lines defined in the header.txt. In the following cases, header line will not be changed. - No header.txt in the current directory. - No header line at the first line of the file. - No header line entry in header.txt. ### Additional options These additional options cannot be used alone but together with other process options. (input-data-path)= #### Specify input data path ``` csv_checker ........ -p ``` Specify relative or absolute path in <path>. If the path name contains white space, the whole path should be enclosed by double quotation marks. ("") (rebuild-time-data)= #### Rebuild time data ``` csv_checker ........ -o ``` This option will be used together with dividing file option (-d), extracting rows options (-t) or thinning out data option (-e). Time data of the output data file will be rebuild incrementally from 1 instead of using the time information of the corresponding line. This feature assumes that time data is in the first columns. If it is not, the conversion will not properly work. (force-process)= #### Force data processing ``` csv_checker ........ -f ``` If there are errors detected by format check such as Error2, Error8, or undefined, processing data is usually not possible without correcting the format first. This option forces to process the data by ignoring the errors. In that case, the area where there is an error will not be modified and keep the data as it is.