如何在火花数据框列中保留前导零
[英]How to retain leading zero in spark data frame column
问题描述
Here is my data
这是我的数据
Source.organizationId|^|Source.sourceId|^|FilingDateTime|^|SourceTypeCode|^|DocumentId|^|Dcn|^|DocFormat|^|StatementDate|^|IsFilingDateTimeEstimated|^|ContainsPreliminaryData|^|CapitalChangeAdjustmentDate|^|CumulativeAdjustmentFactor|^|ContainsRestatement|^|FilingDateTimeUTCOffset|^|ThirdPartySourceCode|^|ThirdPartySourcePriority|^|SourceTypeId|^|ThirdPartySourceCodeId|^|FFAction|!|
4295858966|^|62|^|2017-08-31T11:00:00Z|^|AUTINR|^|44819867|^|cr06305583|^|PDFNTV|^|2017-06-30T00:00:00Z|^|False|^|False|^|2017-06-30T00:00:00Z|^|1.00000|^|False|^|60|^|SS |^|1|^|3011930|^|1000716240|^|I|!|
For the column CumulativeAdjustmentFactor value is 1.00000 .对于列CumulativeAdjustmentFactor值是1.00000 。 When i load and print it i get当我加载并打印它时，我得到
1.0
How to retain the same value?如何保留相同的值？
Schema is below架构如下
root
|-- Source_organizationId: long (nullable = true)
|-- Source_sourceId: integer (nullable = true)
|-- FilingDateTime: string (nullable = true)
|-- SourceTypeCode: string (nullable = true)
|-- DocumentId: integer (nullable = true)
|-- Dcn: string (nullable = true)
|-- DocFormat: string (nullable = true)
|-- StatementDate: string (nullable = true)
|-- IsFilingDateTimeEstimated: boolean (nullable = true)
|-- ContainsPreliminaryData: boolean (nullable = true)
|-- CapitalChangeAdjustmentDate: string (nullable = true)
|-- CumulativeAdjustmentFactor: double (nullable = true)
|-- ContainsRestatement: boolean (nullable = true)
|-- FilingDateTimeUTCOffset: integer (nullable = true)
|-- ThirdPartySourceCode: string (nullable = true)
|-- ThirdPartySourcePriority: integer (nullable = true)
|-- SourceTypeId: integer (nullable = true)
|-- ThirdPartySourceCodeId: integer (nullable = true)
|-- FFAction: string (nullable = true)
|-- DataPartition: string (nullable = true)
When i print i get below output当我打印时，我得到低于输出
+---------------------+---------------+--------------------+--------------+----------+--------+---------+--------------------+-------------------------+-----------------------+---------------------------+--------------------------+-------------------+-----------------------+--------------------+------------------------+------------+----------------------+--------+-------------+
|Source_organizationId|Source_sourceId| FilingDateTime|SourceTypeCode|DocumentId| Dcn|DocFormat| StatementDate|IsFilingDateTimeEstimated|ContainsPreliminaryData|CapitalChangeAdjustmentDate|CumulativeAdjustmentFactor|ContainsRestatement|FilingDateTimeUTCOffset|ThirdPartySourceCode|ThirdPartySourcePriority|SourceTypeId|ThirdPartySourceCodeId|FFAction|DataPartition|
+---------------------+---------------+--------------------+--------------+----------+--------+---------+--------------------+-------------------------+-----------------------+---------------------------+--------------------------+-------------------+-----------------------+--------------------+------------------------+------------+----------------------+--------+-------------+
| 4295880476| 228|2016-00-28T03:00:00Z| JPNARA| null| null| null|2016-00-31T12:00:00Z| false| false| 2016-00-31T12:00:00Z| 1.0| false| 540| SS | 1| 3012415| 1000716240| I| Japan|
| 4295876886| 312|2014-00-25T12:00:00Z| JPNARA| null| null| null|2014-00-31T12:00:00Z| false| false| 2015-00-31T12:00:00Z| 5.0| false| 0| SS | 1| 3012415| 1000716240| I| Japan|
| 4295878485| 12|2001-00-14T12:00:00Z| JPNINR| null| null| null|2001-00-30T12:00:00Z| false| false| 2001-00-30T12:00:00Z| 0.5| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295878485| 17|2002-00-14T12:00:00Z| JPNINR| null| null| null|2002-00-30T12:00:00Z| false| false| 2002-00-30T12:00:00Z| 0.5| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295878485| 46|2006-00-26T12:00:00Z| JPNINR| null| null| null|2005-00-31T12:00:00Z| false| false| 2005-00-31T12:00:00Z| 0.5| true| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295876080| 114|2012-00-10T12:00:00Z| JPNINR| null| null| null|2012-00-01T12:00:00Z| false| false| 2012-00-01T12:00:00Z| 0.5| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295880487| 7|2001-00-22T12:00:00Z| JPNINR| null| null| null|2001-00-30T12:00:00Z| false| false| 2001-00-30T12:00:00Z| null| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295880487| 9|2002-00-14T12:00:00Z| JPNINR| null| null| null|2002-00-30T12:00:00Z| false| false| 2002-00-30T12:00:00Z| null| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295880487| 10|2002-00-20T12:00:00Z| JPNINR| null| null| null|2002-00-30T12:00:00Z| false| false| 2002-00-30T12:00:00Z| null| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295880534| 128|2012-00-04T12:00:00Z| JPNINR| null| null| null|2012-00-01T12:00:00Z| false| false| 2012-00-01T12:00:00Z| 1.0| false| 0| SS | 1| 3012418| 1000716240| I| Japan|
| 4295877413| 173|2014-00-28T03:00:00Z| 20F| 34601135|14723203| ASFILED|2013-00-31T12:00:00Z| false| false| 2013-00-31T12:00:00Z| 1.0| false| 540| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 21|2003-00-30T12:00:00Z| 20F| 1210145| null| null|2003-00-31T12:00:00Z| false| false| 2003-00-31T12:00:00Z| 0.5| false| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 28|2004-00-29T12:00:00Z| 20F| 3866028| null| null|2004-00-31T12:00:00Z| false| false| 2004-00-31T12:00:00Z| 0.5| false| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 38|2005-00-29T12:00:00Z| 20F| 5538099| null| null|2005-00-31T12:00:00Z| false| false| 2005-00-31T12:00:00Z| 0.5| false| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 49|2006-00-28T12:00:00Z| 20F| 7351897| null| null|2006-00-31T12:00:00Z| false| false| 2006-00-31T12:00:00Z| 0.5| false| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 55|2007-00-28T12:00:00Z| 20F| 8834460| null| null|2007-00-31T12:00:00Z| false| false| 2007-00-31T12:00:00Z| null| false| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 177|2015-00-25T03:00:00Z| 20F| 38086474|15950616| ASFILED|2015-00-31T12:00:00Z| false| false| 2015-00-31T12:00:00Z| 1.0| false| 540| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 11|2001-00-04T12:00:00Z| 20F| null| null| null|2001-00-31T12:00:00Z| false| false| 2001-00-31T12:00:00Z| 0.5| true| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 16|2002-00-20T12:00:00Z| 20F| 3556586| null| null|2002-00-31T12:00:00Z| false| false| 2002-00-31T12:00:00Z| 0.5| false| 0| SS | 1| 3011845| 1000716240| I| Japan|
| 4295878485| 57|2007-00-25T12:00:00Z| 6K0| 8923021| null| null|2007-00-30T12:00:00Z| false| false| 2007-00-30T12:00:00Z| null| false| 0| SS | 1| 3011851| 1000716240| I| Japan|
+---------------------+---------------+--------------------+--------------+----------+--------+---------+--------------------+-------------------------+-----------------------+---------------------------+--------------------------+-------------------+-----------------------+--------------------+------------------------+------------+----------------------+--------+-------------+
2 个解决方案
解决方案1
0 2017-11-30 04:07:06
It can not be done because your CumulativeAdjustmentFactor column type is double , the show method simply call the toString of double .不能这样做，因为您的CumulativeAdjustmentFactor列类型是double ， show方法只是调用double的toString 。
> 1.0000.toString
1.0
It's inherent behavior of double .这是double的固有行为。
You can convert to string type to display full value.您可以转换为string类型以显示完整值。
解决方案2
0 2017-11-30 05:08:33
If you want to convert it to a string, there's the format_number method that's discussed here with examples.如果您想将其转换为字符串，请使用此处通过示例讨论的 format_number 方法。 This method takes a number for the precision you want, and will retains trailing zeros in the pyspark implementation, scala spark I haven't tested it though.这个方法需要一个你想要的精度的数字，并且会在 pyspark 实现中保留尾随零，虽然我没有测试过 scala spark。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.
相关问题
枢轴旋转后，spark保留原始数据框的所有列 - spark retain all the columns of orginal data frame after pivot
如何在 Spark (Scala) 中获取前导零的整数值 - How to get integer value with leading zero in Spark (Scala)
如何在scala spark中添加指定位数的前导零填充？ - How to add leading zero padding with the specified number of digits in scala spark?
如何在Spark中对数据框的列执行二进制“或”操作 - How to perform binary “or” operations on column of Data Frame in Spark
Apache Spark：如何将带有正则表达式的数据框列转换为另一个数据框？ - Apache Spark: how to transform Data Frame column with regex to another Data Frame?
如何通过在Spark中将一个大数据帧连接到许多小数据帧来执行列级验证 - How to perform column level validation by joining one Big data frame to many small data frame in spark
在 Apache Spark 1.3 中将一列附加到数据帧 - Append a column to Data Frame in Apache Spark 1.3
在 spark csv 数据框中删除列 - Drop column(s) in spark csv data frame
在Spark 2.2.0和Scala中对数据框的列求和 - Sum the column of a data frame in Spark 2.2.0 and Scala
拆分并过滤Spark中的数据框列 - Split and filter the column of data frame in Spark
相关标签