How to exclude headers and footers while importing multiple txt files via Import Multiple Files Wizard?

Question

I am trying to import multiple *.txt data files from folders and subfolders. All data files have headers and footers (both starting with !).

Headers are from line 1 to line 21. Line 22 has the test parameter names while test data starts from line 23. When data ends, the footer information starts (with !). All data is tab separated.

1. I am unable to get data imported with parameter names as column names using settings as per numbers, even when I try for a single data file. The header line is always showing up as column name and parameters in the first data row.

2. I do not expect to be able to remove the footer information with my settings, but I need to. How to achieve this?

Al the end, I need to vertically concatenate the data with an additional file-name column and correct column (parameter) names. Where am I going wrong?

If this is not possible via the multiple file import wizard, what are my options via JSL?

jthi · Answer

As you have not provided example file, I assume that the "file" I copy pasted is what you have

You should be able to ignore headers by just utilizing settings, for footers, I think you will have to remove them afterwards

Names Default To Here(1);

dt = Multiple File Import(
	<<Set Folder("$DOWNLOADS\843237\"),
	<<Set Add File Name Column(1),
	<<Set Import Mode("CSVData"),
	<<Set Charset("Best Guess"),
	<<Set Stack Mode("Stack Similar"),
	<<Set CSV Has Headers(1),
	<<Set CSV Allow Numeric(1),
	<<Set CSV First Header Line(3),
	<<Set CSV Number Of Header Lines(1),
	<<Set CSV First Data Line(4),
	<<Set CSV EOF Comma(0),
	<<Set CSV EOF Tab(0),
	<<Set CSV EOF Space(0),
	<<Set CSV EOF Spaces(1),
	<<Set CSV EOF Other(""),
	<<Set CSV EOL CRLF(1),
	<<Set CSV EOL CR(1),
	<<Set CSV EOL LF(1),
	<<Set CSV EOL Semicolon(0),
	<<Set CSV EOL Other(""),
	<<Set CSV Quote("\!""),
	<<Set CSV Escape(""),

) << Import Data;

dt = dt[1];

rows_to_delete = dt << Get Rows Where(Starts With(:Parameter, "! "));
dt << Delete Rows(rows_to_delete);

jthi · Answer

Are header and footer data useless information?

Neo · Answer

@jthi Thanks. Here

https://community.jmp.com/t5/Discussions/How-to-exclude-header-and-footer-when-using-open-to-import-a/td-p/443851

I had given an example earlier of how my data file looks like only now I have hundreds of them in folders and subfolder within.

Also, they are *.txt as mentioned in my OP (and not a JMP data table), hence I went the Import Multiple Files way.

Ideally, I would like to use the Import Multiple Files wizard, but perhaps invoke it via a JSL to take into account my requirements. Could I get some direction please?