Extract javascript variables using Python regex

Question

I want to scrape some data from javascript in a website. I have used Beautiful Soup to extract all the comments of the webpage. However, since I could not find any good Javascript parser for Python, I am planning to extract the variables of interest and then eventually use eval to convert them to Python variables. However, I am not good at regular expressions and unable to write the code to extract the JS arrays.

Here is a sample of the data:

var FirstColumnInGrid = "partnername";
var strFlow4Display = "Sell";
var strProduct4Display = "Product";

var strLanguage = "en";
var col0 = ["World","Albania","Antigua and Barbuda","Argentina","Aruba","Australia","Austria","Bahrain","Barbados","Belgium","Belize","Bermuda","Bolivia","Bosnia and Herzegovina","Brazil","Canada","Cayman Islands","Chile","China","Colombia","Cote d'Ivoire","Croatia","Cuba","Curaçao","Cyprus","Denmark","Dominica","Dominican Republic","East Asia & Pacific","Ecuador","Egypt, Arab Rep.","El Salvador","Ethiopia(excludes Eritrea)","Europe & Central Asia","Finland","France","Gabon","Georgia","Germany","Ghana","Greece","Grenada","Guam","Guatemala","Guyana","Haiti","Honduras","Hong Kong, China","India","Indonesia","North America","Norway","Oman","Other Asia, nes","Pakistan","Panama","Paraguay","Peru","Philippines","Poland","Portugal","Qatar","Romania"];
var col1 = ["2865381.57","82.56","95.26","212.68","1275.13","11229.17","2.92","39.09","4.06","211279.63","4.06","15.84","33.98","13.45","126.10","20007.63","1.21","6995.31","4220.84","1459.76","73.12","222.49","281.74","83.09","27.48","4308.36","94.65","2127.18","49178.45","352.75","43.11"];
var col2 = ["28.92","68.08","19.23","2.79","6.56","46.53","0.53","95.64","0.04","39.70","0.04","4.72","1.52","97.82","0.37","23.41","0.23","20.23","9.28","2.08","24.65","67.17","0.87","2.63","32.37","96.81","99.52","57.44","1.52","98.16","9.41","100.00","1.64","7.91","0.30","1.25","0.15","1.04","0.25","99.66","24.53","81.02","92.67","3.24","12.87","4.91","87.97","68.20","44.46","22.99","100.00","6.59","19.10","1.94","100.00","96.63","59.02","40.89"];
var col3 = ["10.34","5.98","4.60","0.02","","10.17","","2.87","0.34","10.13","0.00","0.93","","15.98","0.06","17.06","","6.34","0.20","0.29","","3.87","","","7.00","22.05","","0.17","","0.22","7.30","0.63","0.33",""];
for (var i = 0; i < 116; i++) { var row = {};row["col0"] = col0[i];
row["col1"] = col1[i];
row["col2"] = col2[i];
row["col3"] = col3[i];
row["col4"] = col4[i];
row["col5"] = col5[i];
partData[i] = row; }

I want to extract the arrays col1, col2, ... . I have used the regex:

col\d = 

to match the first part, but I cannot seem to get it to match till the end, that is, till ] .

I have tried using:

col\d = \[*\]

But, it says that it is an invalid expression.

Specifically, I want to extract the arrays as:

col0 = ["World","Albania","Antigua and Barbuda","Argentina","Aruba","Australia","Austria","Bahrain","Barbados","Belgium","Belize","Bermuda","Bolivia","Bosnia and Herzegovina","Brazil","Canada","Cayman Islands","Chile","China","Colombia","Cote d'Ivoire","Croatia","Cuba","Curaçao","Cyprus","Denmark","Dominica","Dominican Republic","East Asia & Pacific","Ecuador","Egypt, Arab Rep.","El Salvador","Ethiopia(excludes Eritrea)","Europe & Central Asia","Finland","France","Gabon","Georgia","Germany","Ghana","Greece","Grenada","Guam","Guatemala","Guyana","Haiti","Honduras","Hong Kong, China","India","Indonesia","North America","Norway","Oman","Other Asia, nes","Pakistan","Panama","Paraguay","Peru","Philippines","Poland","Portugal","Qatar","Romania"]

so that I can potentially use eval to convert them to Python lists.

Answer 1

What you are doing right now does not mean the right thing. You're probably using a wildcard * in the hope of matching whatever comes between two brackets \\[*\\] but in Regular Expression world * depends on the previous pattern so \\[* means zero or more occurrences of opening bracket.

What you are looking for is:

col\d = \[[^\]]*\]

I replaced * with [^\\]]* which for engine means try to match anything but a closing bracket.

See live demo here