how to fix UnicodeEncodeError:?

Question

so i'm trying to scrap data about motherboard from a local website.

import bs4
import os
import requests

from bs4 import BeautifulSoup as soup

os.chdir('E://')
os.makedirs('E://scrappy', exist_ok=True)
myurl = "https://www.example.com"
res = requests.get(myurl)
page = soup(res.content, 'html.parser')
containers = page.findAll("div", {"class": "content-product"})
filename = 'AM4.csv'
f = open(filename, 'w')
headers = 'Motherboard_Name, Price\n'
f.write(headers)

for container in containers:
    Product = container.findAll("div", {"class": "product-title"})
    Motherboard_Name = Product[0].text.strip()
    Kimat = container.findAll("span", {"class": "price"})
    Price = Kimat[0].text
    print('Motherboard_Name' + Motherboard_Name)
    print('Price' + Price)
    f.write(Motherboard_Name + "," + Price.replace(",", "") + "\n")
f.close() print("done")

But when i run this code i get an error

UnicodeEncodeError:'charmap' codec can't encode character '\₹' in position 45: character maps to

how can i fix this??

Edit:: So i fixed the unicode error by adding encoding="utf-8" ( as it was mentioned here python 3.2 UnicodeEncodeError: 'charmap' codec can't encode character '\–' in position 9629: character maps to <undefined> ) (open(filename, 'w',encoding="utf-8" ))and it seems to do the work however in the csv file m getting characters like ( ₹ ) before the price.. How can i fix this?

Answer 1

Use the csv module to manage CSV files, and use utf-8-sig for Excel to recognize UTF-8 properly. Make sure to use newline='' per the csv documentation when opening the file as well.

Example:

import csv

filename = 'AM4.csv'
with open(filename,'w',newline='',encoding='utf-8-sig') as f:
    w = csv.writer(f)
    w.writerow(['Motherboard_Name','Price'])
    name = 'some name'
    price = '\u20b95,99'
    w.writerow([name,price.replace(',','')])