3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
EVAL1_SPC1
application UI
Details for Stage 364 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
6 s
Locality Level Summary:
Process local: 26
Input Size / Records:
5.8 MiB / 29428
Shuffle Write Size / Records:
8.9 MiB / 617988
Associated Job Ids:
190
DAG Visualization
digraph G { subgraph clusterstage_364 { label="Stage 364"; subgraph cluster4533 { label="Scan parquet "; 1780 [labelType="html" label="FileScanRDD [1780]<br>takeAsList at DataFrameCommonJob.java:145"]; 1781 [labelType="html" label="MapPartitionsRDD [1781]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4529 { label="WholeStageCodegen (1)"; 1782 [labelType="html" label="MapPartitionsRDD [1782]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4543 { label="Scan parquet "; 1786 [labelType="html" label="FileScanRDD [1786]<br>takeAsList at DataFrameCommonJob.java:145"]; 1787 [labelType="html" label="MapPartitionsRDD [1787]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4534 { label="WholeStageCodegen (2)"; 1788 [labelType="html" label="MapPartitionsRDD [1788]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4606 { label="Scan parquet "; 1789 [labelType="html" label="FileScanRDD [1789]<br>takeAsList at DataFrameCommonJob.java:145"]; 1790 [labelType="html" label="MapPartitionsRDD [1790]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4546 { label="WholeStageCodegen (3)"; 1791 [labelType="html" label="MapPartitionsRDD [1791]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4611 { label="Scan parquet "; 1792 [labelType="html" label="FileScanRDD [1792]<br>takeAsList at DataFrameCommonJob.java:145"]; 1793 [labelType="html" label="MapPartitionsRDD [1793]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4607 { label="WholeStageCodegen (4)"; 1794 [labelType="html" label="MapPartitionsRDD [1794]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4616 { label="Scan parquet "; 1795 [labelType="html" label="FileScanRDD [1795]<br>takeAsList at DataFrameCommonJob.java:145"]; 1796 [labelType="html" label="MapPartitionsRDD [1796]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4612 { label="WholeStageCodegen (5)"; 1797 [labelType="html" label="MapPartitionsRDD [1797]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4621 { label="Scan parquet "; 1798 [labelType="html" label="FileScanRDD [1798]<br>takeAsList at DataFrameCommonJob.java:145"]; 1799 [labelType="html" label="MapPartitionsRDD [1799]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4617 { label="WholeStageCodegen (6)"; 1800 [labelType="html" label="MapPartitionsRDD [1800]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4626 { label="Scan parquet "; 1801 [labelType="html" label="FileScanRDD [1801]<br>takeAsList at DataFrameCommonJob.java:145"]; 1802 [labelType="html" label="MapPartitionsRDD [1802]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4622 { label="WholeStageCodegen (7)"; 1803 [labelType="html" label="MapPartitionsRDD [1803]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4631 { label="Scan parquet "; 1804 [labelType="html" label="FileScanRDD [1804]<br>takeAsList at DataFrameCommonJob.java:145"]; 1805 [labelType="html" label="MapPartitionsRDD [1805]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4627 { label="WholeStageCodegen (8)"; 1806 [labelType="html" label="MapPartitionsRDD [1806]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4636 { label="Scan parquet "; 1807 [labelType="html" label="FileScanRDD [1807]<br>takeAsList at DataFrameCommonJob.java:145"]; 1808 [labelType="html" label="MapPartitionsRDD [1808]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4632 { label="WholeStageCodegen (9)"; 1809 [labelType="html" label="MapPartitionsRDD [1809]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4641 { label="Scan parquet "; 1810 [labelType="html" label="FileScanRDD [1810]<br>takeAsList at DataFrameCommonJob.java:145"]; 1811 [labelType="html" label="MapPartitionsRDD [1811]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4637 { label="WholeStageCodegen (10)"; 1812 [labelType="html" label="MapPartitionsRDD [1812]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4646 { label="Scan parquet "; 1813 [labelType="html" label="FileScanRDD [1813]<br>takeAsList at DataFrameCommonJob.java:145"]; 1814 [labelType="html" label="MapPartitionsRDD [1814]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4642 { label="WholeStageCodegen (11)"; 1815 [labelType="html" label="MapPartitionsRDD [1815]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4651 { label="Scan parquet "; 1816 [labelType="html" label="FileScanRDD [1816]<br>takeAsList at DataFrameCommonJob.java:145"]; 1817 [labelType="html" label="MapPartitionsRDD [1817]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4647 { label="WholeStageCodegen (12)"; 1818 [labelType="html" label="MapPartitionsRDD [1818]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4656 { label="Scan parquet "; 1819 [labelType="html" label="FileScanRDD [1819]<br>takeAsList at DataFrameCommonJob.java:145"]; 1820 [labelType="html" label="MapPartitionsRDD [1820]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4652 { label="WholeStageCodegen (13)"; 1821 [labelType="html" label="MapPartitionsRDD [1821]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4528 { label="Union"; 1822 [labelType="html" label="UnionRDD [1822]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4524 { label="WholeStageCodegen (14)"; 1823 [labelType="html" label="MapPartitionsRDD [1823]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster4523 { label="Exchange"; 1824 [labelType="html" label="MapPartitionsRDD [1824]<br>takeAsList at DataFrameCommonJob.java:145"]; } } 1780->1781; 1781->1782; 1786->1787; 1787->1788; 1789->1790; 1790->1791; 1792->1793; 1793->1794; 1795->1796; 1796->1797; 1798->1799; 1799->1800; 1801->1802; 1802->1803; 1804->1805; 1805->1806; 1807->1808; 1808->1809; 1810->1811; 1811->1812; 1813->1814; 1814->1815; 1816->1817; 1817->1818; 1819->1820; 1820->1821; 1782->1822; 1788->1822; 1791->1822; 1794->1822; 1797->1822; 1800->1822; 1803->1822; 1806->1822; 1809->1822; 1812->1822; 1815->1822; 1818->1822; 1821->1822; 1822->1823; 1823->1824; }
Event Timeline
Enable zooming
Tasks: 26. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time