Quantcast
Channel: SCN : All Content - Scripting Languages
Viewing all articles
Browse latest Browse all 522

Tip: How To Concatenate Data Files and Transform It To CSV Files

$
0
0

Hello community,

 

I presented here the possibility how to convert CSV files and here how to change the format to JSON or XML.

 

Last but not least here two scripts how to create CSV files from data files. In my case I get a lot of data files with the same structure of the content and with file name patterns, e.g. 456789.deliverylist, where the supplier code is 456789. First of all I must concatenate all delivery lists into one file. The first script shows how to do that. You must define your file name pattern and the function adds all files with this name pattern into one file.

The second step is to transform the data files into CSV files, this means to add a headline with the field names and to set the semicolons at the right position. In my case the data files contains only the data, no separators and no structure information.

 

Concat data files

 

//-Begin----------------------------------------------------------------  //-Directives---------------------------------------------------------    #Option Explicit    #AppType Console  //-Includes-----------------------------------------------------------    #Include <Include/Windows.inc>  //-Sub Concat---------------------------------------------------------    Sub Concat(FilePattern As String)      //-Variables------------------------------------------------------        Static WFD As String * 318        Dim hFileFind As Integer        Dim FileName As String        Dim hFile As Integer        Dim Buffer As String      If FindFirst(FilePattern, @hFileFind, @WFD) <> "" Then        Do          StrCpy(FileName, GetMem(WFD, 44, $MAX_PATH))          hFile = FileOpen(FileName, Binary_Input)          If hFile Then            Buffer = Buffer & FileGet(hFile, FileLen(FileName))            FileClose(hFile)          End If        Loop While FindNext(@hFileFind, @WFD) <> ""      End If      If Buffer <> "" Then        Buffer = Replace(Buffer, CrLf, Lf)        Buffer = Replace(Buffer, Lf, CrLf)        FileName = Left(FilePattern, Len(FilePattern) - 2)        hFile = FileOpen(FileName, Binary_New)        If hFile Then          FilePut(hFile, Buffer)          FileClose(hFile)        End If      End If    End Sub  //-Sub Main-----------------------------------------------------------    Sub Main()      //-Variables------------------------------------------------------        Dim FileNamePart As String      FileNamePart = "4711"      Concat("Individual1." & FileNamePart & ".*")      Concat("Individual2." & FileNamePart & ".*")      Pause    End Sub

//-End------------------------------------------------------------------

 

Transform data files to CSV

 

//-Begin----------------------------------------------------------------  //-Directives---------------------------------------------------------    #Option Explicit    #AppType Console  //-Includes-----------------------------------------------------------    #Include <Include/Windows.inc>  //-Sub Concat---------------------------------------------------------    Sub CreateCSV(FileName As String)      //-Variables------------------------------------------------------        Dim hFile As Integer        Dim hNewFile As Integer        Dim FilePart[]        Dim Line As String        Dim NewLine As String      hFile = FileOpen(FileName, Input)      If hFile Then        FilePart = Split(FileName, ".")        Select Case FilePart[0]          Case "Individual1"            hNewFile = FileOpen(FilePart[0] & ".csv", Output)            If hNewFile Then              //-Add Headline-------------------------------------------                FilePrint(hNewFile, UCase("field1;field2;field3;" & _                  "field4;field5;field6;field7;field8;field9;"))              Do                Line = FileInput(hFile)                //-Add comma at defined position------------------------                  If Line <> "" Then                    NewLine = Mid(Line, 1, 10) & ";"                    NewLine = NewLine & Mid(Line, 11, 9) & ";"                    NewLine = NewLine & Mid(Line, 20, 8) & ";"                    NewLine = NewLine & Mid(Line, 28, 10) & ";"                    NewLine = NewLine & Mid(Line, 38, 10) & ";"                    NewLine = NewLine & Mid(Line, 48, 10) & ";"                    NewLine = NewLine & Mid(Line, 58, 10) & ";"                    NewLine = NewLine & Mid(Line, 68, 10) & ";"                    NewLine = NewLine & Mid(Line, 78, 10) & ";"                    FilePrint(hNewFile, NewLine)                  End If              Loop Until FileEof(hFile)              FileClose(hNewFile)            End If          Case "Individual2"            hNewFile = FileOpen(FilePart[0] & ".csv", Output)            If hNewFile Then              //-Add Headline-------------------------------------------                FilePrint(hNewFile, UCase("field1;field2;field3;" & _                  "field4;field5;field6;field7;field8;field9;" & _                  "field10;field11;"))              Do                Line = FileInput(hFile)                //-Add comma at defined position------------------------                If Line <> "" Then                  NewLine = Mid(Line, 1, 10) & ";"                  NewLine = NewLine & Mid(Line, 11, 10) & ";"                  NewLine = NewLine & Mid(Line, 21, 10) & ";"                  NewLine = NewLine & Mid(Line, 31, 10) & ";"                  NewLine = NewLine & Mid(Line, 41, 10) & ";"                  NewLine = NewLine & Mid(Line, 51, 10) & ";"                  NewLine = NewLine & Mid(Line, 61, 10) & ";"                  NewLine = NewLine & Mid(Line, 71, 10) & ";"                  NewLine = NewLine & Mid(Line, 81, 10) & ";"                  NewLine = NewLine & Mid(Line, 91, 8) & ";"                  NewLine = NewLine & Mid(Line, 98, 1) & ";"                  FilePrint(hNewFile, NewLine)                End If              Loop Until FileEof(hFile)              FileClose(hNewFile)            End If        End Select        FileClose(hFile)      End If    End Sub  //-Sub Main-----------------------------------------------------------    Sub Main()      //-Variables------------------------------------------------------        Dim FileNamePart As String      FileNamePart = "4711"      CreateCSV("Individual1." & FileNamePart)      CreateCSV("Individual2." & FileNamePart)      Pause    End Sub

//-End------------------------------------------------------------------

 

As you can see, good old fashioned CSV files offers even today a lot of potential. With these two scripts and this routine you can simulate a data collection environment on your presentation server with a scripting language very easily.

 

Enjoy it.

 

Cheers

Stefan


Viewing all articles
Browse latest Browse all 522

Latest Images

Trending Articles



Latest Images

<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>