3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
EVAL1_SPC1
application UI
Details for Stage 247 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 26
Input Size / Records:
5.8 MiB / 29428
Shuffle Write Size / Records:
3.2 MiB / 617988
Associated Job Ids:
139
DAG Visualization
digraph G { subgraph clusterstage_247 { label="Stage 247"; subgraph cluster3052 { label="Scan parquet "; 1204 [labelType="html" label="FileScanRDD [1204]<br>takeAsList at DataFrameCommonJob.java:145"]; 1205 [labelType="html" label="MapPartitionsRDD [1205]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3048 { label="WholeStageCodegen (1)"; 1206 [labelType="html" label="MapPartitionsRDD [1206]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3057 { label="Scan parquet "; 1207 [labelType="html" label="FileScanRDD [1207]<br>takeAsList at DataFrameCommonJob.java:145"]; 1208 [labelType="html" label="MapPartitionsRDD [1208]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3053 { label="WholeStageCodegen (2)"; 1209 [labelType="html" label="MapPartitionsRDD [1209]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3062 { label="Scan parquet "; 1210 [labelType="html" label="FileScanRDD [1210]<br>takeAsList at DataFrameCommonJob.java:145"]; 1211 [labelType="html" label="MapPartitionsRDD [1211]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3058 { label="WholeStageCodegen (3)"; 1212 [labelType="html" label="MapPartitionsRDD [1212]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3067 { label="Scan parquet "; 1213 [labelType="html" label="FileScanRDD [1213]<br>takeAsList at DataFrameCommonJob.java:145"]; 1214 [labelType="html" label="MapPartitionsRDD [1214]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3063 { label="WholeStageCodegen (4)"; 1215 [labelType="html" label="MapPartitionsRDD [1215]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3072 { label="Scan parquet "; 1216 [labelType="html" label="FileScanRDD [1216]<br>takeAsList at DataFrameCommonJob.java:145"]; 1217 [labelType="html" label="MapPartitionsRDD [1217]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3068 { label="WholeStageCodegen (5)"; 1218 [labelType="html" label="MapPartitionsRDD [1218]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3077 { label="Scan parquet "; 1219 [labelType="html" label="FileScanRDD [1219]<br>takeAsList at DataFrameCommonJob.java:145"]; 1220 [labelType="html" label="MapPartitionsRDD [1220]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3073 { label="WholeStageCodegen (6)"; 1221 [labelType="html" label="MapPartitionsRDD [1221]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3082 { label="Scan parquet "; 1222 [labelType="html" label="FileScanRDD [1222]<br>takeAsList at DataFrameCommonJob.java:145"]; 1223 [labelType="html" label="MapPartitionsRDD [1223]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3078 { label="WholeStageCodegen (7)"; 1224 [labelType="html" label="MapPartitionsRDD [1224]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3087 { label="Scan parquet "; 1225 [labelType="html" label="FileScanRDD [1225]<br>takeAsList at DataFrameCommonJob.java:145"]; 1226 [labelType="html" label="MapPartitionsRDD [1226]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3083 { label="WholeStageCodegen (8)"; 1227 [labelType="html" label="MapPartitionsRDD [1227]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3092 { label="Scan parquet "; 1228 [labelType="html" label="FileScanRDD [1228]<br>takeAsList at DataFrameCommonJob.java:145"]; 1229 [labelType="html" label="MapPartitionsRDD [1229]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3088 { label="WholeStageCodegen (9)"; 1230 [labelType="html" label="MapPartitionsRDD [1230]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3097 { label="Scan parquet "; 1231 [labelType="html" label="FileScanRDD [1231]<br>takeAsList at DataFrameCommonJob.java:145"]; 1232 [labelType="html" label="MapPartitionsRDD [1232]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3093 { label="WholeStageCodegen (10)"; 1233 [labelType="html" label="MapPartitionsRDD [1233]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3102 { label="Scan parquet "; 1234 [labelType="html" label="FileScanRDD [1234]<br>takeAsList at DataFrameCommonJob.java:145"]; 1235 [labelType="html" label="MapPartitionsRDD [1235]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3098 { label="WholeStageCodegen (11)"; 1236 [labelType="html" label="MapPartitionsRDD [1236]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3107 { label="Scan parquet "; 1237 [labelType="html" label="FileScanRDD [1237]<br>takeAsList at DataFrameCommonJob.java:145"]; 1238 [labelType="html" label="MapPartitionsRDD [1238]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3103 { label="WholeStageCodegen (12)"; 1239 [labelType="html" label="MapPartitionsRDD [1239]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3112 { label="Scan parquet "; 1240 [labelType="html" label="FileScanRDD [1240]<br>takeAsList at DataFrameCommonJob.java:145"]; 1241 [labelType="html" label="MapPartitionsRDD [1241]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3108 { label="WholeStageCodegen (13)"; 1242 [labelType="html" label="MapPartitionsRDD [1242]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3047 { label="Union"; 1243 [labelType="html" label="UnionRDD [1243]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3043 { label="WholeStageCodegen (14)"; 1244 [labelType="html" label="MapPartitionsRDD [1244]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster3042 { label="Exchange"; 1245 [labelType="html" label="MapPartitionsRDD [1245]<br>takeAsList at DataFrameCommonJob.java:145"]; } } 1204->1205; 1205->1206; 1207->1208; 1208->1209; 1210->1211; 1211->1212; 1213->1214; 1214->1215; 1216->1217; 1217->1218; 1219->1220; 1220->1221; 1222->1223; 1223->1224; 1225->1226; 1226->1227; 1228->1229; 1229->1230; 1231->1232; 1232->1233; 1234->1235; 1235->1236; 1237->1238; 1238->1239; 1240->1241; 1241->1242; 1206->1243; 1209->1243; 1212->1243; 1215->1243; 1218->1243; 1221->1243; 1224->1243; 1227->1243; 1230->1243; 1233->1243; 1236->1243; 1239->1243; 1242->1243; 1243->1244; 1244->1245; }
Event Timeline
Enable zooming
Tasks: 26. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time