[英]Reading CSVs from a zip file a line at a time
I've got a Spring MVC app with a file upload capability. 我有一个具有文件上传功能的Spring MVC应用程序。 Files are passed to the controller as MultipartFile from which it's easy to get an InputStream.
文件作为MultipartFile传递给控制器,从中很容易获得InputStream。 I'm uploading zip files that contain CSVs and I'm struggling to find a way to open the CSVs and read them a line at a time.
我正在上传包含CSV的zip文件,我正在努力寻找打开CSV并一次读取一行的方法。 There are plenty of examples on the 'net of reading into a fixed sizes buffer.
“读入固定大小缓冲区的网络上有很多例子。 I've tried this, but the buffers don't concatenate very well and it soon gets out of sync and uses a lot of memory:
我试过这个,但缓冲区不能很好地连接,很快就会失去同步并占用大量内存:
ZipEntry entry = input.getNextEntry();
while(entry != null)
{
if (entry.getName().matches("Data/CSV/[a-z]{0,1}[a-z]{0,1}.csv"))
{
final String fullPath = entry.getName();
final String filename = fullPath.substring(fullPath.lastIndexOf('/') + 1);
visitor.startFile(filename);
final StringBuilder fileContent = new StringBuilder();
final byte[] buffer = new byte[1024];
while (input.read(buffer) > 0)
fileContent.append(new String(buffer));
final String[] lines = fileContent.toString().split("\n");
for(String line : lines)
{
final String[] columns = line.split(",");
final String postcode = columns[0].replace(" ", "").replace("\"", "");
if (columns.length > 3)
visitor.location(postcode, "", "");
}
visitor.endFile();
}
entry = input.getNextEntry();
}
There must be a better way that actually works. 必须有一种更好的实际工作方式。
Not clear if this suits your need, but have you tried opencsv ( http://opencsv.sourceforge.net )? 不清楚这是否适合您的需要,但您尝试过opencsv( http://opencsv.sourceforge.net )吗? Their example is really intuitive:
他们的例子非常直观:
CSVReader reader = new CSVReader(new FileReader("yourfile.csv"));
String [] nextLine;
while ((nextLine = reader.readNext()) != null) {
// nextLine[] is an array of values from the line
System.out.println(nextLine[0] + nextLine[1] + "etc...");
}
For your case, all you will need is to wrap the zipped file stream into a buffered reader and pass the reader to create a CSVReader and use it: 对于您的情况,您只需要将压缩文件流包装到缓冲读取器中并传递读取器以创建CSVReader并使用它:
FileInputStream fis = new FileInputStream(file);
GZIPInputStream gis = new GZIPInputStream(fis);
InputStreamReader isr = new InputStreamReader(gis);
BufferedReader br = new BufferedReader(isr);
CSVReader reader = new CSVReader(br);
You could use a BufferedReader
which includes the convenient readLine()
method and wont load the entire contents of the file into memory eg 您可以使用包含方便的
readLine()
方法的BufferedReader
,并且不会将文件的全部内容加载到内存中,例如
BufferedReader in = new BufferedReader(new InputStreamReader(input), 1024);
String line=null;
while((line=br.readLine())!=null) {
String[] columns = line.split(",");
//rest of your code
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.