368971_en-US

liliconcom

Concat data in multiple text files with no headers

I have about 500 text files that I need to combine. All of them have no headers. The values are separated by comma as below. (the format is the same for all files)

 

122,3410732003,,ABC2,P
122,3410732003,,ABC2I,13.6
122,3410732020,,ABC2,P
122,3410732020,,ABC2I,4.13

 

So when I try to use the import multiple files option in JMP with csv settings, it warns me that there will be 500 data tables generated.

And when I tried importing the files using the text (one line per row), all the data is in one column.

Unfortunately, I don't know enough about scripting to write my own script. (I tried the attached script I found on this forum, but couldn't get it to work, I'm obviously doing something wrong!)

Any and all help in this is greatly appreciated!

 

 

 

 

Data TableScriptingRe: Concat data in multiple text files with no headers

Something like this should work:

Names Default To Here(1);

path = "C:\WIP\All text files\"; //note ending \

fileList = Files In Directory(path);

//use first table as "base datatable" to concatenate to
dt = Open(
 path || fileList[1],
 columns(
  New Column("c000001", Numeric, "Continuous", Format("Best", 12)),
  New Column("c000002", Numeric, "Continuous", Format("Best", 12)),
  New Column("c000003", Numeric, "Continuous", Format("Best", 12)),
  New Column("c000004", Character, "Nominal"),
  New Column("c000005", Character, "Nominal")
 ),
 Import Settings(
  End Of Line(CRLF, CR, LF),
  End Of Field(Comma, CSV(0)),
  Strip Quotes(1),
  Use Apostrophe as Quotation Mark(0),
  Use Regional Settings(0),
  Scan Whole File(1),
  Treat empty columns as numeric(0),
  CompressNumericColumns(0),
  CompressCharacterColumns(0),
  CompressAllowListCheck(0),
  Labels(0),
  Column Names Start(1),
  Data Starts(1),
  Lines To Read("All"),
  Year Rule("20xx")
 )
);

//start from 2 because we already have file 1 as dt
For(i = 2, i <= N items(fileList), i++,
 dt_temp = Open(
   path || fileList[i],
   columns(
    New Column("c000001", Numeric, "Continuous", Format("Best", 12)),
    New Column("c000002", Numeric, "Continuous", Format("Best", 12)),
    New Column("c000003", Numeric, "Continuous", Format("Best", 12)),
    New Column("c000004", Character, "Nominal"),
    New Column("c000005", Character, "Nominal")
   ),
   Import Settings(
    End Of Line(CRLF, CR, LF),
    End Of Field(Comma, CSV(0)),
    Strip Quotes(1),
    Use Apostrophe as Quotation Mark(0),
    Use Regional Settings(0),
    Scan Whole File(1),
    Treat empty columns as numeric(0),
    CompressNumericColumns(0),
    CompressCharacterColumns(0),
    CompressAllowListCheck(0),
    Labels(0),
    Column Names Start(1),
    Data Starts(1),
    Lines To Read("All"),
    Year Rule("20xx")
   )
  );
 dt << Concatenate(dt_temp, Append to first table);
 Close(dt_temp, no save);
);