RDD Storage Info for Union :- *(1) ColumnarToRow : +- InMemoryTableScan [fractile#94362813, annual_ret#94362815, annual_resret#94362827, numcos#94362836] : +- InMemoryRelation [fractile#94362813, overall#94362814, annual_ret#94362815, annual_resret#94362827, numcos#94362836], StorageLevel(disk, memory, deserialized, 1 replicas) : +- *(1) Project [CASE WHEN ((fractile#94362738 = NA) OR (fractile#94362738 = null)) THEN null ELSE cast(fractile#94362738 as float) END AS fractile#94362813, CASE WHEN ((overall#94362739 = NA) OR (overall#94362739 = null)) THEN null ELSE cast(overall#94362739 as int) END AS overall#94362814, CASE WHEN ((annual_ret#94362740 = NA) OR (annual_ret#94362740 = null)) THEN null ELSE cast(annual_ret#94362740 as float) END AS annual_ret#94362815, CASE WHEN ((annual_resret#94362741 = NA) OR (annual_resret#94362741 = null)) THEN null ELSE cast(annual_resret#94362741 as float) END AS annual_resret#94362827, CASE WHEN ((numcos#94362742 = NA) OR (numcos#94362742 = null)) THEN null ELSE cast(num...

  • Storage Level: Disk Memory Deserialized 1x Replicated
  • Cached Partitions: 2
  • Total Partitions: 2
  • Memory Size: 648.0 B
  • Disk Size: 0.0 B

Data Distribution on 1 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
167.71.162.247:37891 648.0 B (3.4 GiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

2 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_3624039_1 Memory Deserialized 1x Replicated 16.0 B 0.0 B 167.71.162.247:37891
rdd_3624039_0 Memory Deserialized 1x Replicated 632.0 B 0.0 B 167.71.162.247:37891
Page: