RDD Storage Info for Union :- *(1) ColumnarToRow : +- InMemoryTableScan [fractile#94377110, annual_ret#94377112, annual_resret#94377114, numcos#94377117] : +- InMemoryRelation [fractile#94377110, overall#94377111, annual_ret#94377112, annual_resret#94377114, numcos#94377117], StorageLevel(disk, memory, deserialized, 1 replicas) : +- *(1) Project [CASE WHEN ((fractile#94376991 = NA) OR (fractile#94376991 = null)) THEN null ELSE cast(fractile#94376991 as float) END AS fractile#94377110, CASE WHEN ((overall#94376992 = NA) OR (overall#94376992 = null)) THEN null ELSE cast(overall#94376992 as int) END AS overall#94377111, CASE WHEN ((annual_ret#94376993 = NA) OR (annual_ret#94376993 = null)) THEN null ELSE cast(annual_ret#94376993 as float) END AS annual_ret#94377112, CASE WHEN ((annual_resret#94376994 = NA) OR (annual_resret#94376994 = null)) THEN null ELSE cast(annual_resret#94376994 as float) END AS annual_resret#94377114, CASE WHEN ((numcos#94376995 = NA) OR (numcos#94376995 = null)) THEN null ELSE cast(num...

  • Storage Level: Disk Memory Deserialized 1x Replicated
  • Cached Partitions: 2
  • Total Partitions: 2
  • Memory Size: 648.0 B
  • Disk Size: 0.0 B

Data Distribution on 1 Executors

HostOn Heap Memory UsageOff Heap Memory UsageDisk Usage
167.71.162.247:37891 648.0 B (3.4 GiB Remaining) 0.0 B (0.0 B Remaining) 0.0 B

2 Partitions

Page:
Block Name Storage Level Size in Memory ▴ Size on Disk Executors
rdd_3624582_1 Memory Deserialized 1x Replicated 16.0 B 0.0 B 167.71.162.247:37891
rdd_3624582_0 Memory Deserialized 1x Replicated 632.0 B 0.0 B 167.71.162.247:37891
Page: