Subscribe to this thread
Home - General / All posts - Export table as csv. Export without "" around text values. Possible?
David S
136 post(s)
#09-May-24 06:29

File has 6.2m records x 44 fields, 22 of which are are type nvarchar or varchar. None of the values in the text fields contain a comma. Exported file is ~1.3 Gb. Would be nice if didn't need to post process the exported file to remove the double quotes every time the file is exported!

Dimitri


7,449 post(s)
#10-May-24 08:05

Would be nice if didn't need to post process

Agree 100%, but if the double quotes weren't in there the export wouldn't be what most of the world considers to be a CSV file.

As the CSV topicputs it, CSV is a tradition-based format:

Much of how CSV files are used is a matter of tradition that has become accepted over the centuries since the original introduction of CSV format, about the time of the domestication of the ox, and the introduction of IBM FORTRAN in 1972. Over the years, various traditions and attempts at standardization have emerged...

The specification of text is one of those traditions (also from the topic...):

CSV files use long-accepted traditions for manually specifying what values are supposed to be text. A convention almost universally respected in programs that create or consume CSV files is that any content wrapped with quote characters is to be treated as text.

So Manifold honors that tradition:

Manifold exports CSV files always using comma characters as list delimiters, periods as decimal separators and double quotes as text qualifiers.

I completely agree that given the intrinsic inefficiencies of CSV one could have dataports that exported to more efficient variations that, when an expert realizes his or her text fields don't contain commas, could skip the use of double quotes. In your case that would save 272.8 million characters (6.4 * 22 * 2).

But that would require Manifold to provide guardrails and documentation to save inexpert people from hurting themselves and their colleagues by generating "CSV" files that don't honor the consensus convention.

Suggesting a variation that disregards that consensus is probably a thin case to argue (see the "Requests to Ignore Standards" discussion in the Suggestions page), especially since it's a simple matter for experts who know enough to want such things to use SED or similar to strip out double quotes they don't want. Also, in modern times the notion of using inefficient, plain text formats for bigger data is being displaced by better options for open formats, like SQLite / GPKG. (I realize that might not be possible if you're creating data for some legacy software that is stuck with CSV.)

Manifold User Community Use Agreement Copyright (C) 2007-2021 Manifold Software Limited. All rights reserved.