使用java比較csv數據與oracle數據庫表

Question

我需要將我的 csv 文件數據與 oracle 數據庫表進行比較。 數據包含近 9000 行。 任何鏈接和來源我該怎么做。 我正在使用這個線程，但它在列表字符串中使用了 equals 方法，但這並沒有逐行比較數據 csv 和數據庫表

將 csv 文件與 MySQL 數據庫進行比較

Answer 1

Java？ 我不會說 Java。但是，因為它是一個 Oracle 數據庫，所以我建議使用另一種方法 -外部表。 下面是一個基於 Scott 的示例模式及其DEPT表的示例。 CSV 文件包含“適合”該表的數據，但是 - 我想看看差異。

test_dept.csv文件：

10,ACCOUNTING,NEW YORK
20,SALES,CHICAGO
30,RESEARCH,DALLAS
40,OPERATIONS,BOSTON
50,CIA,LANGLEY

外部表：為了使用它，必須有一個目錄（第 8 行）（指向文件系統目錄的 Oracle object，通常位於數據庫服務器上。它包含 csv 文件（第 18 行））； 將要使用它的用戶必須至少擁有read權限：

SQL> create table dept_ext
  2    (deptno   char(2),
  3     dname    char(20),
  4     loc      char(20)
  5    )
  6  organization external (
  7    type oracle_loader
  8    default directory ext_dir
  9    access parameters (
 10      records delimited by newline
 11      fields terminated by ','
 12      missing field values are null
 13      ( deptno  char(2),
 14        dname   char(20),
 15        loc     char(20)
 16      )
 17    )
 18    location ('test_dept.csv')
 19  )
 20  reject limit unlimited;

Table created.

它看到任何數據嗎？

SQL> select * from dept_ext;

DE DNAME                LOC
-- -------------------- --------------------
10 ACCOUNTING           NEW YORK
20 SALES                CHICAGO
30 RESEARCH             DALLAS
40 OPERATIONS           BOSTON
50 CIA                  LANGLEY

是的，它確實。 “原始” dept表中有什么？

SQL> select * from dept;

    DEPTNO DNAME          LOC
---------- -------------- -------------
        10 ACCOUNTING     NEW YORK
        20 RESEARCH       DALLAS
        30 SALES          CHICAGO
        40 OPERATIONS     BOSTON

好的，那現在呢？ 因為它是一個“表”，所以你可以寫任何你想要的select ，將它加入到其他表中......例如：數據庫表中不存在csv文件中的哪些部門？

SQL> select * from dept_ext
  2  where deptno not in (select deptno from dept);

DE DNAME                LOC
-- -------------------- --------------------
50 CIA                  LANGLEY

如果我加入deptno上的表，部門名稱有什么不同嗎？

SQL> select e.deptno, e.dname, e.loc, d.dname, d.loc
  2  from dept_ext e join dept d on d.deptno = e.deptno
  3                             and trim(d.dname) <> trim(e.dname);

DE DNAME                LOC                  DNAME          LOC
-- -------------------- -------------------- -------------- -------------
20 SALES                CHICAGO              RESEARCH       DALLAS
30 RESEARCH             DALLAS               SALES          CHICAGO

SQL>

等等。 看起來它可能會做你想做的事。

Answer 2

如果您嘗試使用 Java 來執行此操作，代碼將會很長。 但是使用SPL比較一個CSV文件和Oracle數據庫中的一個表很方便，開源的Java package。

假設我們在 Oracle 數據庫中有一張員工表：

CREATE TABLE EMPLOYEE
  (EID NUMBER(8),
  NAME VARCHAR2(255),
  SURNAME VARCHAR2(255),
  GENDER VARCHAR2(255),
  STATE VARCHAR2(255),
  BIRTHDAY DATE,
  HIREDATE DATE,
  DEPT VARCHAR2(255),
  SALARY NUMBER(8)
);

INSERT INTO EMPLOYEE VALUES (1,'Rebecca','Moore','F','California',TIMESTAMP'1974-11-20 00:00:00.0',TIMESTAMP'2005-03-11 00:00:00.0','R&D',7000);
INSERT INTO EMPLOYEE VALUES (2,'Ashley','Wilson','F','New York',TIMESTAMP'1980-07-19 00:00:00.0',TIMESTAMP'2008-03-16 00:00:00.0','Finance',11000);
INSERT INTO EMPLOYEE VALUES (3,'Rachel','Johnson','F','New Mexico',TIMESTAMP'1970-12-17 00:00:00.0',TIMESTAMP'2010-12-01 00:00:00.0','Sales',9000);
INSERT INTO EMPLOYEE VALUES (4,'Emily','Smith','F','Texas',TIMESTAMP'1985-03-07 00:00:00.0',TIMESTAMP'2006-08-15 00:00:00.0','HR',7000);
INSERT INTO EMPLOYEE VALUES (5,'Ashley','Smith','F','Texas',TIMESTAMP'1975-05-13 00:00:00.0',TIMESTAMP'2004-07-30 00:00:00.0','R&D',16000);
INSERT INTO EMPLOYEE VALUES (6,'Matthew','Johnson','M','California',TIMESTAMP'1984-07-07 00:00:00.0',TIMESTAMP'2005-07-07 00:00:00.0','Sales',11000);
INSERT INTO EMPLOYEE VALUES (7,'Alexis','Smith','F','Illinois',TIMESTAMP'1972-08-16 00:00:00.0',TIMESTAMP'2002-08-16 00:00:00.0','Sales',9000);
INSERT INTO EMPLOYEE VALUES (8,'Megan','Wilson','F','California',TIMESTAMP'1979-04-19 00:00:00.0',TIMESTAMP'1984-04-19 00:00:00.0','Marketing',11000);
INSERT INTO EMPLOYEE VALUES (9,'Victoria','Davis','F','Texas',TIMESTAMP'1983-12-07 00:00:00.0',TIMESTAMP'2009-12-07 00:00:00.0','HR',3000);
INSERT INTO EMPLOYEE VALUES (10,'Ryan','Johnson','M','Pennsylvania',TIMESTAMP'1976-03-12 00:00:00.0',TIMESTAMP'2006-03-12 00:00:00.0','R&D',13000);

和一個 CSV 文件 employee.csv：

EID,NAME,SURNAME,GENDER,STATE,BIRTHDAY,HIREDATE,DEPT,SALARY
1,Rebecca,Moore,F,California,1974-11-20 00:00:00,2005-03-11 00:00:00,R&D,7000
3,Rachel,Johnson,F,New Mexico,1970-12-17 00:00:00,2010-12-01 00:00:00,Sales,9000
5,Ashley,Smith,F,Texas,1975-05-13 00:00:00,2004-07-30 00:00:00,R&D,16000
7,Alexis,Smith,F,Illinois,1972-08-16 00:00:00,2002-08-16 00:00:00,Sales,9000
9,Victoria,Davis,F,Texas,1983-12-07 00:00:00,2009-12-07 00:00:00,HR,3000

為了獲得 Oracle 員工表和 CSV 文件之間的差異（以下是預期結果）：

EID,NAME,SURNAME,GENDER,STATE,BIRTHDAY,HIREDATE,DEPT,SALARY
2,Ashley,Wilson,F,New York,1980-07-19 00:00:00,2008-03-16 00:00:00,Finance,11000
4,Emily,Smith,F,Texas,1985-03-07 00:00:00,2006-08-15 00:00:00,HR,7000
6,Matthew,Johnson,M,California,1984-07-07 00:00:00,2005-07-07 00:00:00,Sales,11000
8,Megan,Wilson,F,California,1979-04-19 00:00:00,1984-04-19 00:00:00,Marketing,11000
10,Ryan,Johnson,M,Pennsylvania,1976-03-12 00:00:00,2006-03-12 00:00:00,R&D,13000

並計算 Oracle 員工表與 CSV 文件的交集：

EID,NAME,SURNAME,GENDER,STATE,BIRTHDAY,HIREDATE,DEPT,SALARY
1,Rebecca,Moore,F,California,1974-11-20 00:00:00,2005-03-11 00:00:00,R&D,7000
3,Rachel,Johnson,F,New Mexico,1970-12-17 00:00:00,2010-12-01 00:00:00,Sales,9000
5,Ashley,Smith,F,Texas,1975-05-13 00:00:00,2004-07-30 00:00:00,R&D,16000
7,Alexis,Smith,F,Illinois,1972-08-16 00:00:00,2002-08-16 00:00:00,Sales,9000
9,Victoria,Davis,F,Texas,1983-12-07 00:00:00,2009-12-07 00:00:00,HR,3000

我們只需要幾行 SPL 代碼：

	一種
1個	=ORACLE.query@x("SELECT * FROM EMPLOYEE")
2個	=file("employee.csv").import@ct(EID:decimal,NAME,SURNAME,GENDER,STATE,BIRTHDAY,HIREDATE,DEPT,SALARY:decimal)
3個	=INTERSECT=[A1,A2].merge@oi(EID,NAME,SURNAME,GENDER,STATE,BIRTHDAY,HIREDATE,DEPT,SALARY)
4個	=MINUS=[A1,A2].merge@od(EID,NAME,SURNAME,GENDER,STATE,BIRTHDAY,HIREDATE,DEPT,SALARY)

SPL提供了JDBC驅動，Java可以調用。只需將上面的SPL腳本保存為cmp.splx，在Java調用存儲過程即可：

…
Class.forName("com.esproc.jdbc.InternalDriver");
con= DriverManager.getConnection("jdbc:esproc:local://");
st=con.prepareCall("call cmp()");
st.execute();
…