RDD Storage Info for Union :- *(1) ColumnarToRow : +- InMemoryTableScan [fractile#94348253, annual_ret#94348255, annual_resret#94348256, numcos#94348257] : +- InMemoryRelation [fractile#94348253, overall#94348254, annual_ret#94348255, annual_resret#94348256, numcos#94348257], StorageLevel(disk, memory, deserialized, 1 replicas) : +- *(1) Project [CASE WHEN ((fractile#94348227 = NA) OR (fractile#94348227 = null)) THEN null ELSE cast(fractile#94348227 as float) END AS fractile#94348253, CASE WHEN ((overall#94348228 = NA) OR (overall#94348228 = null)) THEN null ELSE cast(overall#94348228 as int) END AS overall#94348254, CASE WHEN ((annual_ret#94348229 = NA) OR (annual_ret#94348229 = null)) THEN null ELSE cast(annual_ret#94348229 as float) END AS annual_ret#94348255, CASE WHEN ((annual_resret#94348230 = NA) OR (annual_resret#94348230 = null)) THEN null ELSE cast(annual_resret#94348230 as float) END AS annual_resret#94348256, CASE WHEN ((numcos#94348231 = NA) OR (numcos#94348231 = null)) THEN null ELSE cast(num...

  • Storage Level: Disk Memory Deserialized 1x Replicated
  • Cached Partitions: 2
  • Total Partitions: 2
  • Memory Size: 648.0 B
  • Disk Size: 0.0 B

Data Distribution on 1 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
167.71.162.247:37891 648.0 B (3.4 GiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

2 Partitions

Page:
Block Name ▾ Storage Level Size in Memory Size on Disk Executors
rdd_3623502_1 Memory Deserialized 1x Replicated 16.0 B 0.0 B 167.71.162.247:37891
rdd_3623502_0 Memory Deserialized 1x Replicated 632.0 B 0.0 B 167.71.162.247:37891
Page: