I have json, it contains next keys
[u'domain', u'_timestamp', u'meta_tags', u'author', u'title', u'url', u'tags', u'flow', u'link_tags', u'content', u'post_id', u'flags', u'polling', u'published', u'hubs', u'_id']
I need to writelines from it to .vw
file. But some of them are numeric, and some string. And I need to save this types.
Also I have file with values of target
url target
vk.com 0.934250
I use
targets = train_target.target.values.tolist()
with open('train.json') as inp_json, \
open('habr_train.vw', 'w') as out_vw:
for i, line in enumerate(tqdm_notebook(inp_json)):
data_json = json.loads(line)
if data_json['flow'] is None and data_json['author']['nickname'] is None:
res_line = str(targets[i]) + ' |title ' + data_json['title'] + ' |tags ' + ' '.join(data_json['tags']) \
+ ' |domain ' + data_json['domain'] + ' |flow None' + ' |author None' + ' |hubs ' + data_json['hubs'][0]['title'] + ' |num content_len:' + str(round(len(data_json['content']) / 1000000, 1)) + ' month:' + str(datetime.fromtimestamp(data_json['_timestamp']).month) + ' hour:' + str(datetime.fromtimestamp(data_json['_timestamp']).hour) + '\n'
elif data_json['flow'] is None:
res_line = str(targets[i]) + ' |title ' + data_json['title'] + ' |tags ' + ' '.join(data_json['tags']) \
+ ' |domain ' + data_json['domain'] + ' |flow None' + ' |author ' + data_json['author']['nickname'] + ' |hubs ' + data_json['hubs'][0]['title'] + ' |num content_len:' + str(round(len(data_json['content']) / 1000000, 1)) + ' month:' + str(datetime.fromtimestamp(data_json['_timestamp']).month) + ' hour:' + str(datetime.fromtimestamp(data_json['_timestamp']).hour) + '\n'
elif data_json['author']['nickname'] is None:
res_line = str(targets[i]) + ' |title ' + data_json['title'] + ' |tags ' + ' '.join(data_json['tags']) \
+ ' |domain ' + data_json['domain'] + ' |flow ' + data_json['flow'] + ' |author None' + ' |hubs ' + data_json['hubs'][0]['title'] + ' |num content_len:' + str(round(len(data_json['content']) / 1000000, 1)) + ' month:' + str(datetime.fromtimestamp(data_json['_timestamp']).month) + ' hour:' + str(datetime.fromtimestamp(data_json['_timestamp']).hour) + '\n'
else:
res_line = str(targets[i]) + ' |title ' + data_json['title'] + ' |tags ' + ' '.join(data_json['tags']) \
+ ' |domain ' + data_json['domain'] + ' |flow ' + data_json['flow'] + ' |author ' + data_json['author']['nickname'] + ' |hubs ' + data_json['hubs'][0]['title'] + ' |num content_len:' + str(round(len(data_json['content']) / 1000000, 1)) + ' month:' + str(datetime.fromtimestamp(data_json['_timestamp']).month) + ' hour:' + str(datetime.fromtimestamp(data_json['_timestamp']).hour) + '\n'
out_vw.write(res_line.encode('utf-8'))
It works, but next I need to use library and it returns me error, that str(targets[i]) should be float.
Is any way to save types of values? How can I fix that?
instead of use the concatenate operator you can use format
to avoid those kind of errors
example :
res_line = '{0} |title {1} |tags {2} |domain {3} |flow None |author None |hubs {4} |num content_len: {5} month: {6} hour: {7}\n'.format(str(targets[i]),data_json['title'], ' '.join(data_json['tags']), data_json['domain'], data_json['hubs'][0]['title'], str(datetime.fromtimestamp(data_json['_timestamp']).month), str(datetime.fromtimestamp(data_json['_timestamp']).hour))
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.