Project [date#94010643 AS date#94010641, numcos#94010669 AS numcos#94010663]
Filter (isnotnull(cap#94010646) AND (cast(cap#94010646 as string) = 2))
WholeStageCodegen (1)
InMemoryTableScan [cap#94010646, date#94010643, numcos#94010669], [isnotnull(cap#94010646), (cast(cap#94010646 as string) = 2)]
Project [CASE WHEN (date#94010359 = null) THEN null ELSE cast(date#94010359 as date) END AS date#94010643, CASE WHEN ((cap#94010360 = NA) OR (cap#94010360 = null)) THEN null ELSE cast(cap#94010360 as float) END AS cap#94010646, CASE WHEN ((ret#94010361 = NA) OR (ret#94010361 = null)) THEN null ELSE cast(ret#94010361 as float) END AS ret#94010650, CASE WHEN ((resret#94010362 = NA) OR (resret#94010362 = null)) THEN null ELSE cast(resret#94010362 as float) END AS resret#94010656, CASE WHEN ((retnet#94010363 = NA) OR (retnet#94010363 = null)) THEN null ELSE cast(retnet#94010363 as float) END AS retnet#94010660, CASE WHEN ((turnover#94010364 = NA) OR (turnover#94010364 = null)) THEN null ELSE cast(turnover#94010364 as float) END AS turnover#94010664, CASE WHEN ((numcos#94010365 = NA) OR (numcos#94010365 = null)) THEN null ELSE cast(numcos#94010365 as float) END AS numcos#94010669, CASE WHEN ((coverage#94010366 = NA) OR (coverage#94010366 = null)) THEN null ELSE cast(coverage#94010366 as float) END AS coverage#94010673, CASE WHEN (benchmark#94010367 = null) THEN null ELSE benchmark#94010367 END AS benchmark#94010677, CASE WHEN (excess_ret#94010368 = null) THEN null ELSE excess_ret#94010368 END AS excess_ret#94010681, CASE WHEN (excess_resret#94010369 = null) THEN null ELSE excess_resret#94010369 END AS excess_resret#94010683, CASE WHEN (excess_retnet#94010370 = null) THEN null ELSE excess_retnet#94010370 END AS excess_retnet#94010698]
== Physical Plan ==
CollectLimit (11)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- * Sort (10)
+- Exchange (9)
+- * Project (8)
+- * Filter (7)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- * Project (6)
+- Scan csv (5)
(1) InMemoryTableScan
Output [2]: [date#94010641, numcos#94010663]
Arguments: [date#94010641, numcos#94010663]
(2) InMemoryRelation
Arguments: [date#94010641, numcos#94010663], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@208e3fd9,StorageLevel(disk, memory, deserialized, 1 replicas),*(2) Sort [date#94010641 ASC NULLS FIRST], true, 0
+- Exchange rangepartitioning(date#94010641 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [id=#7516160]
+- *(1) Project [date#94010643 AS date#94010641, numcos#94010669 AS numcos#94010663]
+- *(1) Filter (isnotnull(cap#94010646) AND (cast(cap#94010646 as string) = 2))
+- InMemoryTableScan [cap#94010646, date#94010643, numcos#94010669], [isnotnull(cap#94010646), (cast(cap#94010646 as string) = 2)]
+- InMemoryRelation [date#94010643, cap#94010646, ret#94010650, resret#94010656, retnet#94010660, turnover#94010664, numcos#94010669, coverage#94010673, benchmark#94010677, excess_ret#94010681, excess_resret#94010683, excess_retnet#94010698], StorageLevel(disk, memory, deserialized, 1 replicas)
+- *(1) Project [CASE WHEN (date#94010359 = null) THEN null ELSE cast(date#94010359 as date) END AS date#94010643, CASE WHEN ((cap#94010360 = NA) OR (cap#94010360 = null)) THEN null ELSE cast(cap#94010360 as float) END AS cap#94010646, CASE WHEN ((ret#94010361 = NA) OR (ret#94010361 = null)) THEN null ELSE cast(ret#94010361 as float) END AS ret#94010650, CASE WHEN ((resret#94010362 = NA) OR (resret#94010362 = null)) THEN null ELSE cast(resret#94010362 as float) END AS resret#94010656, CASE WHEN ((retnet#94010363 = NA) OR (retnet#94010363 = null)) THEN null ELSE cast(retnet#94010363 as float) END AS retnet#94010660, CASE WHEN ((turnover#94010364 = NA) OR (turnover#94010364 = null)) THEN null ELSE cast(turnover#94010364 as float) END AS turnover#94010664, CASE WHEN ((numcos#94010365 = NA) OR (numcos#94010365 = null)) THEN null ELSE cast(numcos#94010365 as float) END AS numcos#94010669, CASE WHEN ((coverage#94010366 = NA) OR (coverage#94010366 = null)) THEN null ELSE cast(coverage#94010366 as float) END AS coverage#94010673, CASE WHEN (benchmark#94010367 = null) THEN null ELSE benchmark#94010367 END AS benchmark#94010677, CASE WHEN (excess_ret#94010368 = null) THEN null ELSE excess_ret#94010368 END AS excess_ret#94010681, CASE WHEN (excess_resret#94010369 = null) THEN null ELSE excess_resret#94010369 END AS excess_resret#94010683, CASE WHEN (excess_retnet#94010370 = null) THEN null ELSE excess_retnet#94010370 END AS excess_retnet#94010698]
+- FileScan csv [date#94010359,cap#94010360,ret#94010361,resret#94010362,retnet#94010363,turnover#94010364,numcos#94010365,coverage#94010366,benchmark#94010367,excess_ret#94010368,excess_resret#94010369,excess_retnet#94010370] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rank..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<date:string,cap:string,ret:string,resret:string,retnet:string,turnover:string,numcos:strin...
,None), [date#94010641 ASC NULLS FIRST]
(3) InMemoryTableScan
Output [3]: [cap#94010646, date#94010643, numcos#94010669]
Arguments: [cap#94010646, date#94010643, numcos#94010669], [isnotnull(cap#94010646), (cast(cap#94010646 as string) = 2)]
(4) InMemoryRelation
Arguments: [date#94010643, cap#94010646, ret#94010650, resret#94010656, retnet#94010660, turnover#94010664, numcos#94010669, coverage#94010673, benchmark#94010677, excess_ret#94010681, excess_resret#94010683, excess_retnet#94010698], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@208e3fd9,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Project [CASE WHEN (date#94010359 = null) THEN null ELSE cast(date#94010359 as date) END AS date#94010643, CASE WHEN ((cap#94010360 = NA) OR (cap#94010360 = null)) THEN null ELSE cast(cap#94010360 as float) END AS cap#94010646, CASE WHEN ((ret#94010361 = NA) OR (ret#94010361 = null)) THEN null ELSE cast(ret#94010361 as float) END AS ret#94010650, CASE WHEN ((resret#94010362 = NA) OR (resret#94010362 = null)) THEN null ELSE cast(resret#94010362 as float) END AS resret#94010656, CASE WHEN ((retnet#94010363 = NA) OR (retnet#94010363 = null)) THEN null ELSE cast(retnet#94010363 as float) END AS retnet#94010660, CASE WHEN ((turnover#94010364 = NA) OR (turnover#94010364 = null)) THEN null ELSE cast(turnover#94010364 as float) END AS turnover#94010664, CASE WHEN ((numcos#94010365 = NA) OR (numcos#94010365 = null)) THEN null ELSE cast(numcos#94010365 as float) END AS numcos#94010669, CASE WHEN ((coverage#94010366 = NA) OR (coverage#94010366 = null)) THEN null ELSE cast(coverage#94010366 as float) END AS coverage#94010673, CASE WHEN (benchmark#94010367 = null) THEN null ELSE benchmark#94010367 END AS benchmark#94010677, CASE WHEN (excess_ret#94010368 = null) THEN null ELSE excess_ret#94010368 END AS excess_ret#94010681, CASE WHEN (excess_resret#94010369 = null) THEN null ELSE excess_resret#94010369 END AS excess_resret#94010683, CASE WHEN (excess_retnet#94010370 = null) THEN null ELSE excess_retnet#94010370 END AS excess_retnet#94010698]
+- FileScan csv [date#94010359,cap#94010360,ret#94010361,resret#94010362,retnet#94010363,turnover#94010364,numcos#94010365,coverage#94010366,benchmark#94010367,excess_ret#94010368,excess_resret#94010369,excess_retnet#94010370] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rank..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<date:string,cap:string,ret:string,resret:string,retnet:string,turnover:string,numcos:strin...
,None)
(5) Scan csv
Output [12]: [date#94010359, cap#94010360, ret#94010361, resret#94010362, retnet#94010363, turnover#94010364, numcos#94010365, coverage#94010366, benchmark#94010367, excess_ret#94010368, excess_resret#94010369, excess_retnet#94010370]
Batched: false
Location: InMemoryFileIndex [file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rankviews/longshort_cap.csv]
ReadSchema: struct<date:string,cap:string,ret:string,resret:string,retnet:string,turnover:string,numcos:string,coverage:string,benchmark:string,excess_ret:string,excess_resret:string,excess_retnet:string>
(6) Project [codegen id : 1]
Output [12]: [CASE WHEN (date#94010359 = null) THEN null ELSE cast(date#94010359 as date) END AS date#94010643, CASE WHEN ((cap#94010360 = NA) OR (cap#94010360 = null)) THEN null ELSE cast(cap#94010360 as float) END AS cap#94010646, CASE WHEN ((ret#94010361 = NA) OR (ret#94010361 = null)) THEN null ELSE cast(ret#94010361 as float) END AS ret#94010650, CASE WHEN ((resret#94010362 = NA) OR (resret#94010362 = null)) THEN null ELSE cast(resret#94010362 as float) END AS resret#94010656, CASE WHEN ((retnet#94010363 = NA) OR (retnet#94010363 = null)) THEN null ELSE cast(retnet#94010363 as float) END AS retnet#94010660, CASE WHEN ((turnover#94010364 = NA) OR (turnover#94010364 = null)) THEN null ELSE cast(turnover#94010364 as float) END AS turnover#94010664, CASE WHEN ((numcos#94010365 = NA) OR (numcos#94010365 = null)) THEN null ELSE cast(numcos#94010365 as float) END AS numcos#94010669, CASE WHEN ((coverage#94010366 = NA) OR (coverage#94010366 = null)) THEN null ELSE cast(coverage#94010366 as float) END AS coverage#94010673, CASE WHEN (benchmark#94010367 = null) THEN null ELSE benchmark#94010367 END AS benchmark#94010677, CASE WHEN (excess_ret#94010368 = null) THEN null ELSE excess_ret#94010368 END AS excess_ret#94010681, CASE WHEN (excess_resret#94010369 = null) THEN null ELSE excess_resret#94010369 END AS excess_resret#94010683, CASE WHEN (excess_retnet#94010370 = null) THEN null ELSE excess_retnet#94010370 END AS excess_retnet#94010698]
Input [12]: [date#94010359, cap#94010360, ret#94010361, resret#94010362, retnet#94010363, turnover#94010364, numcos#94010365, coverage#94010366, benchmark#94010367, excess_ret#94010368, excess_resret#94010369, excess_retnet#94010370]
(7) Filter [codegen id : 1]
Input [3]: [cap#94010646, date#94010643, numcos#94010669]
Condition : (isnotnull(cap#94010646) AND (cast(cap#94010646 as string) = 2))
(8) Project [codegen id : 1]
Output [2]: [date#94010643 AS date#94010641, numcos#94010669 AS numcos#94010663]
Input [3]: [cap#94010646, date#94010643, numcos#94010669]
(9) Exchange
Input [2]: [date#94010641, numcos#94010663]
Arguments: rangepartitioning(date#94010641 ASC NULLS FIRST, 200), ENSURE_REQUIREMENTS, [id=#7516160]
(10) Sort [codegen id : 2]
Input [2]: [date#94010641, numcos#94010663]
Arguments: [date#94010641 ASC NULLS FIRST], true, 0
(11) CollectLimit
Input [2]: [date#94010641, numcos#94010663]
Arguments: 1000000