I cannot access the web UI of my containerized spark cluster even if I copy and paste the following Ip address: The cluster that I've built is take ...
I cannot access the web UI of my containerized spark cluster even if I copy and paste the following Ip address: The cluster that I've built is take ...
Data skew is something that hapen offen, that should be detected and treated correctly, I'm able to detect data skew in specific table using a groupby ...
As per the documentation: Shuffle spill (memory) is the size of the deserialized form of the shuffled data in memory. Shuffle spill (disk) i ...
What is the shufflequerystage box that I see in the spark DAGs. How is it different from the excahnge box in the spark stages? ...
I am trying to understand the spark ui and hdfs ui while using pyspark. Following are my properties for the Session that I am running I ran a simpl ...
I work with Spark SQL v2.4.7 on EMR (with YARN). I write Spark Sql queries to perform transformations. Estimating the optimal Shuffle Partitions numb ...
I'm getting "HTTP ERROR 502 Bad Gateway" when I click on a worker link in my standalone Spark UI. Looking at the master logs I can see a corresponding ...
. Answers to this question are eligible for a +50 reputation bounty. Go ...
I am not able to view the spark-ui for databricks jobs executed through notebook activity in Azure datafactory. Does anyone know which permissions ne ...
I want to make a button click after 10 seconds when the UI loads. I've tried below code in the "On load" event, but seems that the timer doesn't work: ...
I have an AWS glue job with Spark UI enabled by following this instruction: Enabling the Spark UI for Jobs The glue job has s3:* access to arn:aws:s3 ...
I have this code snippet that I ran locally in standalone mode using 100 records only: The schema contains 89 columns all having string data type e ...
i am running a job in spark shell job the job is stuck and not starting the code is doing a cross join with filter conditions on a large dataset of ...
I am new to Spark. I have couple of questions regarding the Spark Web UI:- I have seen that Spark can create multiple Jobs for the same applicatio ...
I am doing some analysis on spark sql query execution plans. the execution plans that explain() api prints are not much readable. If we see spark web ...
Default CF Template to create a history server includes creation of security group and IAM role. I removed both and added to select the existing secur ...
I run a Spark Job and try to tune it faster. It is weird that the total uptime is 1.1 hours, but I add up all the job duration. It only takes 25 mins. ...
As per my understanding, there will be one job for each action in Spark. But often I see there are more than one jobs triggered for a single action. I ...
As the picture below, what's the difference between duration, batch duration and processing time in spark UI ? thanks Spark UI Picture ...
This question specifically regards the localhost. I am trying to embed a localhost web page in another localhost web page however it states that this ...