This topic created in 2888 days ago, the information mentioned may be changed or developed.
请问在读取 hdfs 文件的时候，采用分块 chunksize 读取数据，但怎么会把一条数据拆分成多条呢？
with client.read(full_path,encoding='utf-8',chunk_size=10000) as reader:
for piece in reader:
piece=piece.split('\n')
for line in piece:
print(line)
本来数据是 2018-05-01|weorjerjsfj|worjwelfjs|
结果读出来的数据是 2018-05-01|weo
rjerjsfj|worjwelfjs|分别显示了两条记录
with client.read(full_path,encoding='utf-8',chunk_size=10000) as reader:
for piece in reader:
piece=piece.split('\n')
for line in piece:
print(line)
本来数据是 2018-05-01|weorjerjsfj|worjwelfjs|
结果读出来的数据是 2018-05-01|weo
rjerjsfj|worjwelfjs|分别显示了两条记录
1 replies • 2019-04-19 11:06:21 +08:00
 |
1
RmanzzZ Apr 19, 2019
老哥问题解决了吗 遇到同样问题了 不知道怎么处理
|
Select Language