[英]Group Pivot Pandas data from mysql database python
我对 python 很陌生,想在 python 中使用sqlalchemy从我的 MySQL 数据库中读取数据。 如何将数据读入熊猫并使用熊猫数据透视表? 数据库结构如下所示:
Date_String Experiment Experiment_Type RESET_FREQUENCY MEASURE_LENGTH Value Date_Integer
28-Sep-16 A FORWARD_Detector 1 Minute 1 0.99994 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 7 0.99959 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 14 0.99917 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 21 0.99876 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 30 0.99823 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 60 0.99647 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 90 0.99469 20160928
28-Sep-16 A FORWARD_Detector 1 Minute 120 0.99288 20160928
29-Sep-16 A FORWARD_Detector 1 Minute 1 0.99994 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 7 0.99959 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 14 0.99918 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 21 0.99877 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 30 0.99824 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 60 0.99646 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 90 0.99472 20160929
29-Sep-16 A FORWARD_Detector 1 Minute 120 0.99287 20160929
30-Sep-16 A FORWARD_Detector 1 Minute 1 0.99994 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 7 0.99959 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 14 0.99918 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 21 0.99877 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 30 0.99824 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 60 0.99647 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 90 0.99469 20160930
30-Sep-16 A FORWARD_Detector 1 Minute 120 0.99286 20160930
...
代码如下所示:
import sqlalchemy as sqlal
import matplotlib.pyplot as plt
import pandas as pd
mysql_engine = sqlal.create_engine('mysql+mysqlconnector://xxx@localhost/rates data',poolclass=sqlal.pool.NullPool)
mysql_engine.echo = False
mysql_engine.connect()
metadata = sqlal.MetaData()
'''
experiment_data = sqlal.Table('experiment_data', metadata,
sqlal.Column('Date_String', sqlal.Date(), nullable=True),
sqlal.Column('Experiment', sqlal.String(3), nullable=True),
sqlal.Column('Experiment_Type', sqlal.String(8), nullable=True),
sqlal.Column('RESET_FREQUENCY', sqlal.String(3), nullable=True),
sqlal.Column('MEASURE_LENGHT', sqlal.Integer(), nullable=True),
sqlal.Column('Value', sqlal.Float(), nullable=True),
sqlal.Column('Date_Integer', sqlal.Integer(), nullable=True)
)
'''
#print(mysql_engine.table_names())
Data_exp = sqlal.Table('experiment_data', metadata, autoload=True, autoload_with=mysql_engine)
stmt = sqlal.select([Data_exp])
results = mysql_engine.execute(stmt).fetchall()
data_dataframe = pd.DataFrame(results)
mysql_engine.dispose()
# Print the Dataframe
print(data_dataframe)
data_test= pd.pivot_table(Data_IR,index=["Date_String","MEASURE_LENGTH"],values=["Value"])
#optional way to get a pivot table
#data_test= pd.pivot_table(Data_IR,index=["Date_String"],columns=["MEASURE_LENGTH"],values=["Value"])
如何使用生成的数据透视表绘制我的结果,如下图所示?
只需在pivot_table
对象上使用pandas.DataFrame.plot ,指定一个折线图。 另外,在 pivot_table 的列中分配Date_String
并为index保留MEASURE_LENGTH
:
下面包括使用pd.read_table()
重建数据以重现您发布的数据,但可以忽略,因为您从 MySQL 获取表。 还应考虑pandas.read_sql可以读SQLAlchemy的对象。
Data_IR = pandas.read_sql(stmt, con=mysql_engine)
再现数据(数据略有调整,使三个日期的结果不完全相同)
from io import StringIO
import pandas as pd
txt="""
Date_String Experiment Experiment_Type RESET_FREQUENCY MEASURE_LENGTH Value Date_Integer
28-Sep-16 A FORWARD_Detector "1 Minute" 1 0.99974 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 7 0.99939 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 14 0.99897 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 21 0.99856 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 30 0.99803 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 60 0.99627 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 90 0.99449 20160928
28-Sep-16 A FORWARD_Detector "1 Minute" 120 0.99268 20160928
29-Sep-16 A FORWARD_Detector "1 Minute" 1 0.99994 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 7 0.99959 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 14 0.99918 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 21 0.99877 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 30 0.99824 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 60 0.99646 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 90 0.99472 20160929
29-Sep-16 A FORWARD_Detector "1 Minute" 120 0.99287 20160929
30-Sep-16 A FORWARD_Detector "1 Minute" 1 0.99954 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 7 0.99919 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 14 0.99878 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 21 0.99837 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 30 0.99784 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 60 0.99607 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 90 0.99429 20160930
30-Sep-16 A FORWARD_Detector "1 Minute" 120 0.99246 20160930
"""
Data_IR = pd.read_table(StringIO(txt), sep="\\s+")
阴谋
import matplotlib.pyplot as plt
data_test= pd.pivot_table(Data_IR,index=["MEASURE_LENGTH"], columns=["Date_String"], values="Value")
data_test.plot(kind='line')
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.