I need to extract only unique sublists based on first element from a nested list. For e.g. My method is two break list into two lists and check for ...
I have column with search queries that are represented by strings. I want to separate every string to different work. Let say I have this data frame: ...
I have a very large 3D array (say 100 x 100 x 10) that I would like to apply a function over for pairwise comparisons. I've tried a number of solution ...
I need to check my solution for idempotency and check how much it's different with past solution. I tried next: It's gives me information how much ...
I want a generic query to get fill rate of all columns in table .Query should work irrespective of the column number.I have to implement this using pr ...
I have a table A with the structure: id key value 1 "abc" "123" 1 "d ...
Lets assume that I have an input DataStream and want to implement some functionality that requires "memory" so I need ProcessFunction that gives me ac ...
I have a question and I am wondering if anyone has solved this problem effectively. I am developing a collector(let's call it A) to collect data from ...
I'm working with a table similar to this in bigquery at my job: We want to take this data and perform the following transformation: For each uniqu ...
When indexing in local Vespa, the indexing is slow. My configuration: ` ` and schema: I use /document/v1 API to push documents into Vespa (POS ...
I have a large pandas DataFrame consisting of 1 million rows, and I want to get the Levenshtein distance between every entity in one column of the Dat ...
I am creating a Flink application that reads strings from a Kafka topic for example "2 5 9" is a value. Then split the string with " " delimiter and c ...
I am new to Scala. I am trying to unit test ASSERTIONS for UT/DQ check for Scala Spark Dataframe using ZIO library. Can anyone help me out here if the ...
The JSON file in question is quite large (~1.5GB) but has some metadata at a known location (.meta.view.approvals) near the beginning. How can jq or ...
I used a GridSearchCV pipeline for training several different image classifiers in scikit-learn. In the pipeline I used two stages, scaler and classif ...
The Problem On a server, I host ids in a json file. From clients, I need to mandate the server to intersect and sometimes negate these ids (the ids n ...
Hello I am working on a project where I have to pull data between 2018 and 2023. It's about 200 million records (not that many), But now I am confused ...
I would like to use dataset in pyspark . I read pyspark doesnt support dataset and only Java / Scala support dataset . Is there any way I can use Dat ...
I have some metrics data like below, it's Map[String, Any], I want to get the data from Map, e.g. I want to get non_unique -> 1 from metrics data. ...
I have a dataset (Excel file) includes three fields of District (string) , Land Use (string), and Temperature (numeric). By the way the overall number ...