3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
EVAL1_SPC1
application UI
Details for Stage 288 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 26
Input Size / Records:
5.8 MiB / 29428
Shuffle Write Size / Records:
3.2 MiB / 617988
Associated Job Ids:
156
DAG Visualization
digraph G { subgraph clusterstage_288 { label="Stage 288"; subgraph cluster3584 { label="Scan parquet "; 1410 [labelType="html" label="FileScanRDD [1410]<br>takeAsList at DataFrameCommonJob.java:145"]; 1411 [labelType="html" label="MapPartitionsRDD [1411]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3580 { label="WholeStageCodegen (1)"; 1412 [labelType="html" label="MapPartitionsRDD [1412]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3589 { label="Scan parquet "; 1413 [labelType="html" label="FileScanRDD [1413]<br>takeAsList at DataFrameCommonJob.java:145"]; 1414 [labelType="html" label="MapPartitionsRDD [1414]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3585 { label="WholeStageCodegen (2)"; 1415 [labelType="html" label="MapPartitionsRDD [1415]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3594 { label="Scan parquet "; 1416 [labelType="html" label="FileScanRDD [1416]<br>takeAsList at DataFrameCommonJob.java:145"]; 1417 [labelType="html" label="MapPartitionsRDD [1417]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3590 { label="WholeStageCodegen (3)"; 1418 [labelType="html" label="MapPartitionsRDD [1418]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3599 { label="Scan parquet "; 1419 [labelType="html" label="FileScanRDD [1419]<br>takeAsList at DataFrameCommonJob.java:145"]; 1420 [labelType="html" label="MapPartitionsRDD [1420]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3595 { label="WholeStageCodegen (4)"; 1421 [labelType="html" label="MapPartitionsRDD [1421]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3604 { label="Scan parquet "; 1422 [labelType="html" label="FileScanRDD [1422]<br>takeAsList at DataFrameCommonJob.java:145"]; 1423 [labelType="html" label="MapPartitionsRDD [1423]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3600 { label="WholeStageCodegen (5)"; 1424 [labelType="html" label="MapPartitionsRDD [1424]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3609 { label="Scan parquet "; 1425 [labelType="html" label="FileScanRDD [1425]<br>takeAsList at DataFrameCommonJob.java:145"]; 1426 [labelType="html" label="MapPartitionsRDD [1426]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3605 { label="WholeStageCodegen (6)"; 1427 [labelType="html" label="MapPartitionsRDD [1427]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3614 { label="Scan parquet "; 1428 [labelType="html" label="FileScanRDD [1428]<br>takeAsList at DataFrameCommonJob.java:145"]; 1429 [labelType="html" label="MapPartitionsRDD [1429]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3610 { label="WholeStageCodegen (7)"; 1430 [labelType="html" label="MapPartitionsRDD [1430]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3619 { label="Scan parquet "; 1431 [labelType="html" label="FileScanRDD [1431]<br>takeAsList at DataFrameCommonJob.java:145"]; 1432 [labelType="html" label="MapPartitionsRDD [1432]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3615 { label="WholeStageCodegen (8)"; 1433 [labelType="html" label="MapPartitionsRDD [1433]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3624 { label="Scan parquet "; 1434 [labelType="html" label="FileScanRDD [1434]<br>takeAsList at DataFrameCommonJob.java:145"]; 1435 [labelType="html" label="MapPartitionsRDD [1435]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3620 { label="WholeStageCodegen (9)"; 1436 [labelType="html" label="MapPartitionsRDD [1436]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3629 { label="Scan parquet "; 1437 [labelType="html" label="FileScanRDD [1437]<br>takeAsList at DataFrameCommonJob.java:145"]; 1438 [labelType="html" label="MapPartitionsRDD [1438]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3625 { label="WholeStageCodegen (10)"; 1439 [labelType="html" label="MapPartitionsRDD [1439]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3634 { label="Scan parquet "; 1440 [labelType="html" label="FileScanRDD [1440]<br>takeAsList at DataFrameCommonJob.java:145"]; 1441 [labelType="html" label="MapPartitionsRDD [1441]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3630 { label="WholeStageCodegen (11)"; 1442 [labelType="html" label="MapPartitionsRDD [1442]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3639 { label="Scan parquet "; 1443 [labelType="html" label="FileScanRDD [1443]<br>takeAsList at DataFrameCommonJob.java:145"]; 1444 [labelType="html" label="MapPartitionsRDD [1444]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3635 { label="WholeStageCodegen (12)"; 1445 [labelType="html" label="MapPartitionsRDD [1445]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3644 { label="Scan parquet "; 1446 [labelType="html" label="FileScanRDD [1446]<br>takeAsList at DataFrameCommonJob.java:145"]; 1447 [labelType="html" label="MapPartitionsRDD [1447]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3640 { label="WholeStageCodegen (13)"; 1448 [labelType="html" label="MapPartitionsRDD [1448]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3579 { label="Union"; 1449 [labelType="html" label="UnionRDD [1449]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3575 { label="WholeStageCodegen (14)"; 1450 [labelType="html" label="MapPartitionsRDD [1450]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3574 { label="Exchange"; 1451 [labelType="html" label="MapPartitionsRDD [1451]<br>takeAsList at DataFrameCommonJob.java:145"]; } } 1410->1411; 1411->1412; 1413->1414; 1414->1415; 1416->1417; 1417->1418; 1419->1420; 1420->1421; 1422->1423; 1423->1424; 1425->1426; 1426->1427; 1428->1429; 1429->1430; 1431->1432; 1432->1433; 1434->1435; 1435->1436; 1437->1438; 1438->1439; 1440->1441; 1441->1442; 1443->1444; 1444->1445; 1446->1447; 1447->1448; 1412->1449; 1415->1449; 1418->1449; 1421->1449; 1424->1449; 1427->1449; 1430->1449; 1433->1449; 1436->1449; 1439->1449; 1442->1449; 1445->1449; 1448->1449; 1449->1450; 1450->1451; }
Event Timeline
Enable zooming
Tasks: 26. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time