简体   繁体   English

将数据透视表模板从csv(Excel)移到MySQL

[英]Moving Pivot Table template from csv(Excel) to MySQL

I usually prepare reports and charts from excel manually using pivot table adding several columns manually from the raw data and then using pivot table on the fields and populating it. 我通常使用数据透视表从原始数据手动添加几列,然后在字段上使用数据透视表并填充它,从而手动从excel准备报告和图表。

And I would like to see if this can be automated by: 我想看看是否可以通过以下方式实现自动化:

a) Loading the data into a mysql database a)将数据加载到mysql数据库中
b) Using several queries to add additional columns and then prepare the data ready to be used by b)使用几个查询来添加其他列,然后准备可供数据使用的数据
c) Chart APIs/JQuery. c)图表API / JQuery。

Since I know csv to mysql is easier, I now have the raw data file in CSV format. 由于我知道将csv转换为mysql更容易,因此现在有了CSV格式的原始数据文件。

The raw data basically contains different fields mainly time, date time and strings. 原始数据基本上包含不同的字段,主要是时间,日期时间和字符串。

Using a PHP script, I was able to load these data using the LOAD DATA LOCAL INFILE command. 使用PHP脚本,我能够使用LOAD DATA LOCAL INFILE命令加载这些数据。

Based on dates, I need to prepare a column y which says months and this month column has to be updated with the month name('jan', etc.) depending on the date field (yyyy-mm-dd hh:mm:ss ) on certain x column in the same table. 根据日期,我需要准备一个表示月份的列y,并且必须根据日期字段(yyyy-mm-dd hh:mm:ss )用月份名称(“ jan”等)更新本月列)放在同一表格的某些x列上。

or maybe just use this and reference in the graphs(Not sure how complex that would be):- 或者也许只是使用它并在图表中引用(不确定会有多复杂):

mysql> select count(*) as Count,  monthname(date) from alerts;
+-------+---------------------------------+
| Count | monthname(date) |
+-------+---------------------------------+
| 24124 | March                           |
+-------+---------------------------------+
1 row in set (0.19 sec)

Similarly, I need a column a that says "Duration < 5 minutes" and a column b that says "Duration > 5 min < 10 min" , where I would put a numeric value '1', if it falls within the range. 同样,我需要a列表示“持续时间<5分钟”和b列显示“持续时间> 5分钟<10分钟”,如果数值在该范围内,我将在其中放置数字值“ 1”。

I looked into the self-join examples but I could not make it work in my case inspite of several efforts. 我研究了自连接示例,但是尽管付出了很多努力,但在我的案例中却无法使它起作用。

I need some help to get me going because my belief is that a table with all relevant columns is better off than using queries at runtime. 我需要一些帮助,因为我坚信与所有相关列相关的表比在运行时使用查询要好。 Also, is it better to format the data first and load it to mysql OR load the data and format it? 另外,最好先格式化数据并将其加载到mysql或加载数据并格式化吗? Please let me know. 请告诉我。

Thanks 谢谢

Update1 UPDATE1

Okay, I got this working with a self join as below 好的,我通过如下方式进行了自我加入

UPDATE t1 p1 INNER JOIN ( select monthname(dt_received) AS EXTMONTHNAME from t1)p2 SET     p1.MONTH=p2.EXTMONTHNAME;

but why does it update all the month with the same month name even though dt_received has other months ? 但是,即使dt_received还有其他月份,为什么还要用相同的月份名称更新所有月份? Can someone help? 有人可以帮忙吗?

Update2 Again, still struggling, I was made aware of the 1093 error/constraint. Update2同样,我仍然在挣扎中,意识到了1093错误/约束。 The workarounds are simply not helping 解决方法根本无济于事

Unlike Excel where manual formatting was required, I found querying the database much easier using queryies 与需要手动格式化的Excel不同,我发现使用queryies查询数据库要容易得多

This resolved the issue 这解决了问题

UPDATE tablename p1 INNER JOIN ( select monthname(dt_received) AS EXTMONTHNAME from tablename )p2 SET p1.MONTH=p2.EXTMONTHNAME where monthname(p1.dt_received)=p2.EXTMONTHNAME;

But would someone know, why it takes close to 14 minutes to change 36879 rows? 但是有人会知道,为什么要花费近14分钟才能更改36879行? How do I optimize it. 我如何优化它。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM