== Physical Plan ==
CollectLimit (17)
+- * ColumnarToRow (16)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- * Project (15)
+- * BroadcastNestedLoopJoin Cross BuildRight (14)
:- * Project (8)
: +- * Filter (7)
: +- InMemoryTableScan (3)
: +- InMemoryRelation (4)
: +- * Project (6)
: +- Scan csv (5)
+- BroadcastExchange (13)
+- * Project (12)
+- * Filter (11)
+- InMemoryTableScan (9)
+- InMemoryRelation (10)
+- * Project (6)
+- Scan csv (5)
(1) InMemoryTableScan
Output [3]: [ret_large#94356777, ret_small#94356807, differential#94356893]
Arguments: [ret_large#94356777, ret_small#94356807, differential#94356893]
(2) InMemoryRelation
Arguments: [ret_large#94356777, ret_small#94356807, differential#94356893], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@208e3fd9,StorageLevel(disk, memory, deserialized, 1 replicas),*(2) Project [ret_large#94356777, ret_small#94356807, (ret_large#94356777 - ret_small#94356807) AS differential#94356893]
+- *(2) BroadcastNestedLoopJoin BuildRight, Cross
:- *(2) Project [annual_ret#94354242 AS ret_large#94356777]
: +- *(2) Filter (isnotnull(cap#94354236) AND (cap#94354236 = 1.0))
: +- InMemoryTableScan [annual_ret#94354242, cap#94354236], [isnotnull(cap#94354236), (cap#94354236 = 1.0)]
: +- InMemoryRelation [cap#94354236, retIC#94354237, resretIC#94354238, numcos#94354239, numdates#94354240, annual_bmret#94354241, annual_ret#94354242, std_ret#94354243, Sharpe_ret#94354244, PctPos_ret#94354262, TR_ret#94354263, IR_ret#94354264, annual_resret#94354276, std_resret#94354277, Sharpe_resret#94354280, PctPos_resret#94354281, TR_resret#94354282, IR_resret#94354283, annual_retnet#94354284, std_retnet#94354285, Sharpe_retnet#94354286, PctPos_retnet#94354287, TR_retnet#94354288, IR_retnet#94354289, ... 2 more fields], StorageLevel(disk, memory, deserialized, 1 replicas)
: +- *(1) Project [CASE WHEN ((cap#94353974 = NA) OR (cap#94353974 = null)) THEN null ELSE cast(cap#94353974 as float) END AS cap#94354236, CASE WHEN ((retIC#94353975 = NA) OR (retIC#94353975 = null)) THEN null ELSE cast(retIC#94353975 as float) END AS retIC#94354237, CASE WHEN ((resretIC#94353976 = NA) OR (resretIC#94353976 = null)) THEN null ELSE cast(resretIC#94353976 as float) END AS resretIC#94354238, CASE WHEN ((numcos#94353977 = NA) OR (numcos#94353977 = null)) THEN null ELSE cast(numcos#94353977 as float) END AS numcos#94354239, CASE WHEN ((numdates#94353978 = NA) OR (numdates#94353978 = null)) THEN null ELSE cast(numdates#94353978 as int) END AS numdates#94354240, CASE WHEN (annual_bmret#94353979 = null) THEN null ELSE annual_bmret#94353979 END AS annual_bmret#94354241, CASE WHEN ((annual_ret#94353980 = NA) OR (annual_ret#94353980 = null)) THEN null ELSE cast(annual_ret#94353980 as float) END AS annual_ret#94354242, CASE WHEN ((std_ret#94353981 = NA) OR (std_ret#94353981 = null)) THEN null ELSE cast(std_ret#94353981 as float) END AS std_ret#94354243, CASE WHEN ((Sharpe_ret#94353982 = NA) OR (Sharpe_ret#94353982 = null)) THEN null ELSE cast(Sharpe_ret#94353982 as float) END AS Sharpe_ret#94354244, CASE WHEN ((PctPos_ret#94353983 = NA) OR (PctPos_ret#94353983 = null)) THEN null ELSE cast(PctPos_ret#94353983 as float) END AS PctPos_ret#94354262, CASE WHEN (TR_ret#94353984 = null) THEN null ELSE TR_ret#94353984 END AS TR_ret#94354263, CASE WHEN (IR_ret#94353985 = null) THEN null ELSE IR_ret#94353985 END AS IR_ret#94354264, CASE WHEN ((annual_resret#94353986 = NA) OR (annual_resret#94353986 = null)) THEN null ELSE cast(annual_resret#94353986 as float) END AS annual_resret#94354276, CASE WHEN ((std_resret#94353987 = NA) OR (std_resret#94353987 = null)) THEN null ELSE cast(std_resret#94353987 as float) END AS std_resret#94354277, CASE WHEN ((Sharpe_resret#94353988 = NA) OR (Sharpe_resret#94353988 = null)) THEN null ELSE cast(Sharpe_resret#94353988 as float) END AS Sharpe_resret#94354280, CASE WHEN ((PctPos_resret#94353989 = NA) OR (PctPos_resret#94353989 = null)) THEN null ELSE cast(PctPos_resret#94353989 as float) END AS PctPos_resret#94354281, CASE WHEN (TR_resret#94353990 = null) THEN null ELSE TR_resret#94353990 END AS TR_resret#94354282, CASE WHEN (IR_resret#94353991 = null) THEN null ELSE IR_resret#94353991 END AS IR_resret#94354283, CASE WHEN ((annual_retnet#94353992 = NA) OR (annual_retnet#94353992 = null)) THEN null ELSE cast(annual_retnet#94353992 as float) END AS annual_retnet#94354284, CASE WHEN ((std_retnet#94353993 = NA) OR (std_retnet#94353993 = null)) THEN null ELSE cast(std_retnet#94353993 as float) END AS std_retnet#94354285, CASE WHEN ((Sharpe_retnet#94353994 = NA) OR (Sharpe_retnet#94353994 = null)) THEN null ELSE cast(Sharpe_retnet#94353994 as float) END AS Sharpe_retnet#94354286, CASE WHEN ((PctPos_retnet#94353995 = NA) OR (PctPos_retnet#94353995 = null)) THEN null ELSE cast(PctPos_retnet#94353995 as float) END AS PctPos_retnet#94354287, CASE WHEN (TR_retnet#94353996 = null) THEN null ELSE TR_retnet#94353996 END AS TR_retnet#94354288, CASE WHEN (IR_retnet#94353997 = null) THEN null ELSE IR_retnet#94353997 END AS IR_retnet#94354289, ... 2 more fields]
: +- FileScan csv [cap#94353974,retIC#94353975,resretIC#94353976,numcos#94353977,numdates#94353978,annual_bmret#94353979,annual_ret#94353980,std_ret#94353981,Sharpe_ret#94353982,PctPos_ret#94353983,TR_ret#94353984,IR_ret#94353985,annual_resret#94353986,std_resret#94353987,Sharpe_resret#94353988,PctPos_resret#94353989,TR_resret#94353990,IR_resret#94353991,annual_retnet#94353992,std_retnet#94353993,Sharpe_retnet#94353994,PctPos_retnet#94353995,TR_retnet#94353996,IR_retnet#94353997,... 2 more fields] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rank..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<cap:string,retIC:string,resretIC:string,numcos:string,numdates:string,annual_bmret:string,...
+- BroadcastExchange IdentityBroadcastMode, [id=#7533728]
+- *(1) Project [annual_ret#94354242 AS ret_small#94356807]
+- *(1) Filter (isnotnull(cap#94354236) AND (cap#94354236 = 3.0))
+- InMemoryTableScan [annual_ret#94354242, cap#94354236], [isnotnull(cap#94354236), (cap#94354236 = 3.0)]
+- InMemoryRelation [cap#94354236, retIC#94354237, resretIC#94354238, numcos#94354239, numdates#94354240, annual_bmret#94354241, annual_ret#94354242, std_ret#94354243, Sharpe_ret#94354244, PctPos_ret#94354262, TR_ret#94354263, IR_ret#94354264, annual_resret#94354276, std_resret#94354277, Sharpe_resret#94354280, PctPos_resret#94354281, TR_resret#94354282, IR_resret#94354283, annual_retnet#94354284, std_retnet#94354285, Sharpe_retnet#94354286, PctPos_retnet#94354287, TR_retnet#94354288, IR_retnet#94354289, ... 2 more fields], StorageLevel(disk, memory, deserialized, 1 replicas)
+- *(1) Project [CASE WHEN ((cap#94353974 = NA) OR (cap#94353974 = null)) THEN null ELSE cast(cap#94353974 as float) END AS cap#94354236, CASE WHEN ((retIC#94353975 = NA) OR (retIC#94353975 = null)) THEN null ELSE cast(retIC#94353975 as float) END AS retIC#94354237, CASE WHEN ((resretIC#94353976 = NA) OR (resretIC#94353976 = null)) THEN null ELSE cast(resretIC#94353976 as float) END AS resretIC#94354238, CASE WHEN ((numcos#94353977 = NA) OR (numcos#94353977 = null)) THEN null ELSE cast(numcos#94353977 as float) END AS numcos#94354239, CASE WHEN ((numdates#94353978 = NA) OR (numdates#94353978 = null)) THEN null ELSE cast(numdates#94353978 as int) END AS numdates#94354240, CASE WHEN (annual_bmret#94353979 = null) THEN null ELSE annual_bmret#94353979 END AS annual_bmret#94354241, CASE WHEN ((annual_ret#94353980 = NA) OR (annual_ret#94353980 = null)) THEN null ELSE cast(annual_ret#94353980 as float) END AS annual_ret#94354242, CASE WHEN ((std_ret#94353981 = NA) OR (std_ret#94353981 = null)) THEN null ELSE cast(std_ret#94353981 as float) END AS std_ret#94354243, CASE WHEN ((Sharpe_ret#94353982 = NA) OR (Sharpe_ret#94353982 = null)) THEN null ELSE cast(Sharpe_ret#94353982 as float) END AS Sharpe_ret#94354244, CASE WHEN ((PctPos_ret#94353983 = NA) OR (PctPos_ret#94353983 = null)) THEN null ELSE cast(PctPos_ret#94353983 as float) END AS PctPos_ret#94354262, CASE WHEN (TR_ret#94353984 = null) THEN null ELSE TR_ret#94353984 END AS TR_ret#94354263, CASE WHEN (IR_ret#94353985 = null) THEN null ELSE IR_ret#94353985 END AS IR_ret#94354264, CASE WHEN ((annual_resret#94353986 = NA) OR (annual_resret#94353986 = null)) THEN null ELSE cast(annual_resret#94353986 as float) END AS annual_resret#94354276, CASE WHEN ((std_resret#94353987 = NA) OR (std_resret#94353987 = null)) THEN null ELSE cast(std_resret#94353987 as float) END AS std_resret#94354277, CASE WHEN ((Sharpe_resret#94353988 = NA) OR (Sharpe_resret#94353988 = null)) THEN null ELSE cast(Sharpe_resret#94353988 as float) END AS Sharpe_resret#94354280, CASE WHEN ((PctPos_resret#94353989 = NA) OR (PctPos_resret#94353989 = null)) THEN null ELSE cast(PctPos_resret#94353989 as float) END AS PctPos_resret#94354281, CASE WHEN (TR_resret#94353990 = null) THEN null ELSE TR_resret#94353990 END AS TR_resret#94354282, CASE WHEN (IR_resret#94353991 = null) THEN null ELSE IR_resret#94353991 END AS IR_resret#94354283, CASE WHEN ((annual_retnet#94353992 = NA) OR (annual_retnet#94353992 = null)) THEN null ELSE cast(annual_retnet#94353992 as float) END AS annual_retnet#94354284, CASE WHEN ((std_retnet#94353993 = NA) OR (std_retnet#94353993 = null)) THEN null ELSE cast(std_retnet#94353993 as float) END AS std_retnet#94354285, CASE WHEN ((Sharpe_retnet#94353994 = NA) OR (Sharpe_retnet#94353994 = null)) THEN null ELSE cast(Sharpe_retnet#94353994 as float) END AS Sharpe_retnet#94354286, CASE WHEN ((PctPos_retnet#94353995 = NA) OR (PctPos_retnet#94353995 = null)) THEN null ELSE cast(PctPos_retnet#94353995 as float) END AS PctPos_retnet#94354287, CASE WHEN (TR_retnet#94353996 = null) THEN null ELSE TR_retnet#94353996 END AS TR_retnet#94354288, CASE WHEN (IR_retnet#94353997 = null) THEN null ELSE IR_retnet#94353997 END AS IR_retnet#94354289, ... 2 more fields]
+- FileScan csv [cap#94353974,retIC#94353975,resretIC#94353976,numcos#94353977,numdates#94353978,annual_bmret#94353979,annual_ret#94353980,std_ret#94353981,Sharpe_ret#94353982,PctPos_ret#94353983,TR_ret#94353984,IR_ret#94353985,annual_resret#94353986,std_resret#94353987,Sharpe_resret#94353988,PctPos_resret#94353989,TR_resret#94353990,IR_resret#94353991,annual_retnet#94353992,std_retnet#94353993,Sharpe_retnet#94353994,PctPos_retnet#94353995,TR_retnet#94353996,IR_retnet#94353997,... 2 more fields] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rank..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<cap:string,retIC:string,resretIC:string,numcos:string,numdates:string,annual_bmret:string,...
,None)
(3) InMemoryTableScan
Output [2]: [annual_ret#94354242, cap#94354236]
Arguments: [annual_ret#94354242, cap#94354236], [isnotnull(cap#94354236), (cap#94354236 = 1.0)]
(4) InMemoryRelation
Arguments: [cap#94354236, retIC#94354237, resretIC#94354238, numcos#94354239, numdates#94354240, annual_bmret#94354241, annual_ret#94354242, std_ret#94354243, Sharpe_ret#94354244, PctPos_ret#94354262, TR_ret#94354263, IR_ret#94354264, annual_resret#94354276, std_resret#94354277, Sharpe_resret#94354280, PctPos_resret#94354281, TR_resret#94354282, IR_resret#94354283, annual_retnet#94354284, std_retnet#94354285, Sharpe_retnet#94354286, PctPos_retnet#94354287, TR_retnet#94354288, IR_retnet#94354289, ... 2 more fields], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@208e3fd9,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Project [CASE WHEN ((cap#94353974 = NA) OR (cap#94353974 = null)) THEN null ELSE cast(cap#94353974 as float) END AS cap#94354236, CASE WHEN ((retIC#94353975 = NA) OR (retIC#94353975 = null)) THEN null ELSE cast(retIC#94353975 as float) END AS retIC#94354237, CASE WHEN ((resretIC#94353976 = NA) OR (resretIC#94353976 = null)) THEN null ELSE cast(resretIC#94353976 as float) END AS resretIC#94354238, CASE WHEN ((numcos#94353977 = NA) OR (numcos#94353977 = null)) THEN null ELSE cast(numcos#94353977 as float) END AS numcos#94354239, CASE WHEN ((numdates#94353978 = NA) OR (numdates#94353978 = null)) THEN null ELSE cast(numdates#94353978 as int) END AS numdates#94354240, CASE WHEN (annual_bmret#94353979 = null) THEN null ELSE annual_bmret#94353979 END AS annual_bmret#94354241, CASE WHEN ((annual_ret#94353980 = NA) OR (annual_ret#94353980 = null)) THEN null ELSE cast(annual_ret#94353980 as float) END AS annual_ret#94354242, CASE WHEN ((std_ret#94353981 = NA) OR (std_ret#94353981 = null)) THEN null ELSE cast(std_ret#94353981 as float) END AS std_ret#94354243, CASE WHEN ((Sharpe_ret#94353982 = NA) OR (Sharpe_ret#94353982 = null)) THEN null ELSE cast(Sharpe_ret#94353982 as float) END AS Sharpe_ret#94354244, CASE WHEN ((PctPos_ret#94353983 = NA) OR (PctPos_ret#94353983 = null)) THEN null ELSE cast(PctPos_ret#94353983 as float) END AS PctPos_ret#94354262, CASE WHEN (TR_ret#94353984 = null) THEN null ELSE TR_ret#94353984 END AS TR_ret#94354263, CASE WHEN (IR_ret#94353985 = null) THEN null ELSE IR_ret#94353985 END AS IR_ret#94354264, CASE WHEN ((annual_resret#94353986 = NA) OR (annual_resret#94353986 = null)) THEN null ELSE cast(annual_resret#94353986 as float) END AS annual_resret#94354276, CASE WHEN ((std_resret#94353987 = NA) OR (std_resret#94353987 = null)) THEN null ELSE cast(std_resret#94353987 as float) END AS std_resret#94354277, CASE WHEN ((Sharpe_resret#94353988 = NA) OR (Sharpe_resret#94353988 = null)) THEN null ELSE cast(Sharpe_resret#94353988 as float) END AS Sharpe_resret#94354280, CASE WHEN ((PctPos_resret#94353989 = NA) OR (PctPos_resret#94353989 = null)) THEN null ELSE cast(PctPos_resret#94353989 as float) END AS PctPos_resret#94354281, CASE WHEN (TR_resret#94353990 = null) THEN null ELSE TR_resret#94353990 END AS TR_resret#94354282, CASE WHEN (IR_resret#94353991 = null) THEN null ELSE IR_resret#94353991 END AS IR_resret#94354283, CASE WHEN ((annual_retnet#94353992 = NA) OR (annual_retnet#94353992 = null)) THEN null ELSE cast(annual_retnet#94353992 as float) END AS annual_retnet#94354284, CASE WHEN ((std_retnet#94353993 = NA) OR (std_retnet#94353993 = null)) THEN null ELSE cast(std_retnet#94353993 as float) END AS std_retnet#94354285, CASE WHEN ((Sharpe_retnet#94353994 = NA) OR (Sharpe_retnet#94353994 = null)) THEN null ELSE cast(Sharpe_retnet#94353994 as float) END AS Sharpe_retnet#94354286, CASE WHEN ((PctPos_retnet#94353995 = NA) OR (PctPos_retnet#94353995 = null)) THEN null ELSE cast(PctPos_retnet#94353995 as float) END AS PctPos_retnet#94354287, CASE WHEN (TR_retnet#94353996 = null) THEN null ELSE TR_retnet#94353996 END AS TR_retnet#94354288, CASE WHEN (IR_retnet#94353997 = null) THEN null ELSE IR_retnet#94353997 END AS IR_retnet#94354289, ... 2 more fields]
+- FileScan csv [cap#94353974,retIC#94353975,resretIC#94353976,numcos#94353977,numdates#94353978,annual_bmret#94353979,annual_ret#94353980,std_ret#94353981,Sharpe_ret#94353982,PctPos_ret#94353983,TR_ret#94353984,IR_ret#94353985,annual_resret#94353986,std_resret#94353987,Sharpe_resret#94353988,PctPos_resret#94353989,TR_resret#94353990,IR_resret#94353991,annual_retnet#94353992,std_retnet#94353993,Sharpe_retnet#94353994,PctPos_retnet#94353995,TR_retnet#94353996,IR_retnet#94353997,... 2 more fields] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rank..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<cap:string,retIC:string,resretIC:string,numcos:string,numdates:string,annual_bmret:string,...
,None)
(5) Scan csv
Output [26]: [cap#94353974, retIC#94353975, resretIC#94353976, numcos#94353977, numdates#94353978, annual_bmret#94353979, annual_ret#94353980, std_ret#94353981, Sharpe_ret#94353982, PctPos_ret#94353983, TR_ret#94353984, IR_ret#94353985, annual_resret#94353986, std_resret#94353987, Sharpe_resret#94353988, PctPos_resret#94353989, TR_resret#94353990, IR_resret#94353991, annual_retnet#94353992, std_retnet#94353993, Sharpe_retnet#94353994, PctPos_retnet#94353995, TR_retnet#94353996, IR_retnet#94353997, turnover#94353998, coverage#94353999]
Batched: false
Location: InMemoryFileIndex [file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rankviews/stats_cap.csv]
ReadSchema: struct<cap:string,retIC:string,resretIC:string,numcos:string,numdates:string,annual_bmret:string,annual_ret:string,std_ret:string,Sharpe_ret:string,PctPos_ret:string,TR_ret:string,IR_ret:string,annual_resret:string,std_resret:string,Sharpe_resret:string,PctPos_resret:string,TR_resret:string,IR_resret:string,annual_retnet:string,std_retnet:string,Sharpe_retnet:string,PctPos_retnet:string,TR_retnet:string,IR_retnet:string,turnover:string,coverage:string>
(6) Project [codegen id : 1]
Output [26]: [CASE WHEN ((cap#94353974 = NA) OR (cap#94353974 = null)) THEN null ELSE cast(cap#94353974 as float) END AS cap#94354236, CASE WHEN ((retIC#94353975 = NA) OR (retIC#94353975 = null)) THEN null ELSE cast(retIC#94353975 as float) END AS retIC#94354237, CASE WHEN ((resretIC#94353976 = NA) OR (resretIC#94353976 = null)) THEN null ELSE cast(resretIC#94353976 as float) END AS resretIC#94354238, CASE WHEN ((numcos#94353977 = NA) OR (numcos#94353977 = null)) THEN null ELSE cast(numcos#94353977 as float) END AS numcos#94354239, CASE WHEN ((numdates#94353978 = NA) OR (numdates#94353978 = null)) THEN null ELSE cast(numdates#94353978 as int) END AS numdates#94354240, CASE WHEN (annual_bmret#94353979 = null) THEN null ELSE annual_bmret#94353979 END AS annual_bmret#94354241, CASE WHEN ((annual_ret#94353980 = NA) OR (annual_ret#94353980 = null)) THEN null ELSE cast(annual_ret#94353980 as float) END AS annual_ret#94354242, CASE WHEN ((std_ret#94353981 = NA) OR (std_ret#94353981 = null)) THEN null ELSE cast(std_ret#94353981 as float) END AS std_ret#94354243, CASE WHEN ((Sharpe_ret#94353982 = NA) OR (Sharpe_ret#94353982 = null)) THEN null ELSE cast(Sharpe_ret#94353982 as float) END AS Sharpe_ret#94354244, CASE WHEN ((PctPos_ret#94353983 = NA) OR (PctPos_ret#94353983 = null)) THEN null ELSE cast(PctPos_ret#94353983 as float) END AS PctPos_ret#94354262, CASE WHEN (TR_ret#94353984 = null) THEN null ELSE TR_ret#94353984 END AS TR_ret#94354263, CASE WHEN (IR_ret#94353985 = null) THEN null ELSE IR_ret#94353985 END AS IR_ret#94354264, CASE WHEN ((annual_resret#94353986 = NA) OR (annual_resret#94353986 = null)) THEN null ELSE cast(annual_resret#94353986 as float) END AS annual_resret#94354276, CASE WHEN ((std_resret#94353987 = NA) OR (std_resret#94353987 = null)) THEN null ELSE cast(std_resret#94353987 as float) END AS std_resret#94354277, CASE WHEN ((Sharpe_resret#94353988 = NA) OR (Sharpe_resret#94353988 = null)) THEN null ELSE cast(Sharpe_resret#94353988 as float) END AS Sharpe_resret#94354280, CASE WHEN ((PctPos_resret#94353989 = NA) OR (PctPos_resret#94353989 = null)) THEN null ELSE cast(PctPos_resret#94353989 as float) END AS PctPos_resret#94354281, CASE WHEN (TR_resret#94353990 = null) THEN null ELSE TR_resret#94353990 END AS TR_resret#94354282, CASE WHEN (IR_resret#94353991 = null) THEN null ELSE IR_resret#94353991 END AS IR_resret#94354283, CASE WHEN ((annual_retnet#94353992 = NA) OR (annual_retnet#94353992 = null)) THEN null ELSE cast(annual_retnet#94353992 as float) END AS annual_retnet#94354284, CASE WHEN ((std_retnet#94353993 = NA) OR (std_retnet#94353993 = null)) THEN null ELSE cast(std_retnet#94353993 as float) END AS std_retnet#94354285, CASE WHEN ((Sharpe_retnet#94353994 = NA) OR (Sharpe_retnet#94353994 = null)) THEN null ELSE cast(Sharpe_retnet#94353994 as float) END AS Sharpe_retnet#94354286, CASE WHEN ((PctPos_retnet#94353995 = NA) OR (PctPos_retnet#94353995 = null)) THEN null ELSE cast(PctPos_retnet#94353995 as float) END AS PctPos_retnet#94354287, CASE WHEN (TR_retnet#94353996 = null) THEN null ELSE TR_retnet#94353996 END AS TR_retnet#94354288, CASE WHEN (IR_retnet#94353997 = null) THEN null ELSE IR_retnet#94353997 END AS IR_retnet#94354289, CASE WHEN ((turnover#94353998 = NA) OR (turnover#94353998 = null)) THEN null ELSE cast(turnover#94353998 as float) END AS turnover#94354290, CASE WHEN ((coverage#94353999 = NA) OR (coverage#94353999 = null)) THEN null ELSE cast(coverage#94353999 as float) END AS coverage#94354291]
Input [26]: [cap#94353974, retIC#94353975, resretIC#94353976, numcos#94353977, numdates#94353978, annual_bmret#94353979, annual_ret#94353980, std_ret#94353981, Sharpe_ret#94353982, PctPos_ret#94353983, TR_ret#94353984, IR_ret#94353985, annual_resret#94353986, std_resret#94353987, Sharpe_resret#94353988, PctPos_resret#94353989, TR_resret#94353990, IR_resret#94353991, annual_retnet#94353992, std_retnet#94353993, Sharpe_retnet#94353994, PctPos_retnet#94353995, TR_retnet#94353996, IR_retnet#94353997, turnover#94353998, coverage#94353999]
(7) Filter [codegen id : 2]
Input [2]: [annual_ret#94354242, cap#94354236]
Condition : (isnotnull(cap#94354236) AND (cap#94354236 = 1.0))
(8) Project [codegen id : 2]
Output [1]: [annual_ret#94354242 AS ret_large#94356777]
Input [2]: [annual_ret#94354242, cap#94354236]
(9) InMemoryTableScan
Output [2]: [annual_ret#94354242, cap#94354236]
Arguments: [annual_ret#94354242, cap#94354236], [isnotnull(cap#94354236), (cap#94354236 = 3.0)]
(10) InMemoryRelation
Arguments: [cap#94354236, retIC#94354237, resretIC#94354238, numcos#94354239, numdates#94354240, annual_bmret#94354241, annual_ret#94354242, std_ret#94354243, Sharpe_ret#94354244, PctPos_ret#94354262, TR_ret#94354263, IR_ret#94354264, annual_resret#94354276, std_resret#94354277, Sharpe_resret#94354280, PctPos_resret#94354281, TR_resret#94354282, IR_resret#94354283, annual_retnet#94354284, std_retnet#94354285, Sharpe_retnet#94354286, PctPos_retnet#94354287, TR_retnet#94354288, IR_retnet#94354289, ... 2 more fields], CachedRDDBuilder(org.apache.spark.sql.execution.columnar.DefaultCachedBatchSerializer@208e3fd9,StorageLevel(disk, memory, deserialized, 1 replicas),*(1) Project [CASE WHEN ((cap#94353974 = NA) OR (cap#94353974 = null)) THEN null ELSE cast(cap#94353974 as float) END AS cap#94354236, CASE WHEN ((retIC#94353975 = NA) OR (retIC#94353975 = null)) THEN null ELSE cast(retIC#94353975 as float) END AS retIC#94354237, CASE WHEN ((resretIC#94353976 = NA) OR (resretIC#94353976 = null)) THEN null ELSE cast(resretIC#94353976 as float) END AS resretIC#94354238, CASE WHEN ((numcos#94353977 = NA) OR (numcos#94353977 = null)) THEN null ELSE cast(numcos#94353977 as float) END AS numcos#94354239, CASE WHEN ((numdates#94353978 = NA) OR (numdates#94353978 = null)) THEN null ELSE cast(numdates#94353978 as int) END AS numdates#94354240, CASE WHEN (annual_bmret#94353979 = null) THEN null ELSE annual_bmret#94353979 END AS annual_bmret#94354241, CASE WHEN ((annual_ret#94353980 = NA) OR (annual_ret#94353980 = null)) THEN null ELSE cast(annual_ret#94353980 as float) END AS annual_ret#94354242, CASE WHEN ((std_ret#94353981 = NA) OR (std_ret#94353981 = null)) THEN null ELSE cast(std_ret#94353981 as float) END AS std_ret#94354243, CASE WHEN ((Sharpe_ret#94353982 = NA) OR (Sharpe_ret#94353982 = null)) THEN null ELSE cast(Sharpe_ret#94353982 as float) END AS Sharpe_ret#94354244, CASE WHEN ((PctPos_ret#94353983 = NA) OR (PctPos_ret#94353983 = null)) THEN null ELSE cast(PctPos_ret#94353983 as float) END AS PctPos_ret#94354262, CASE WHEN (TR_ret#94353984 = null) THEN null ELSE TR_ret#94353984 END AS TR_ret#94354263, CASE WHEN (IR_ret#94353985 = null) THEN null ELSE IR_ret#94353985 END AS IR_ret#94354264, CASE WHEN ((annual_resret#94353986 = NA) OR (annual_resret#94353986 = null)) THEN null ELSE cast(annual_resret#94353986 as float) END AS annual_resret#94354276, CASE WHEN ((std_resret#94353987 = NA) OR (std_resret#94353987 = null)) THEN null ELSE cast(std_resret#94353987 as float) END AS std_resret#94354277, CASE WHEN ((Sharpe_resret#94353988 = NA) OR (Sharpe_resret#94353988 = null)) THEN null ELSE cast(Sharpe_resret#94353988 as float) END AS Sharpe_resret#94354280, CASE WHEN ((PctPos_resret#94353989 = NA) OR (PctPos_resret#94353989 = null)) THEN null ELSE cast(PctPos_resret#94353989 as float) END AS PctPos_resret#94354281, CASE WHEN (TR_resret#94353990 = null) THEN null ELSE TR_resret#94353990 END AS TR_resret#94354282, CASE WHEN (IR_resret#94353991 = null) THEN null ELSE IR_resret#94353991 END AS IR_resret#94354283, CASE WHEN ((annual_retnet#94353992 = NA) OR (annual_retnet#94353992 = null)) THEN null ELSE cast(annual_retnet#94353992 as float) END AS annual_retnet#94354284, CASE WHEN ((std_retnet#94353993 = NA) OR (std_retnet#94353993 = null)) THEN null ELSE cast(std_retnet#94353993 as float) END AS std_retnet#94354285, CASE WHEN ((Sharpe_retnet#94353994 = NA) OR (Sharpe_retnet#94353994 = null)) THEN null ELSE cast(Sharpe_retnet#94353994 as float) END AS Sharpe_retnet#94354286, CASE WHEN ((PctPos_retnet#94353995 = NA) OR (PctPos_retnet#94353995 = null)) THEN null ELSE cast(PctPos_retnet#94353995 as float) END AS PctPos_retnet#94354287, CASE WHEN (TR_retnet#94353996 = null) THEN null ELSE TR_retnet#94353996 END AS TR_retnet#94354288, CASE WHEN (IR_retnet#94353997 = null) THEN null ELSE IR_retnet#94353997 END AS IR_retnet#94354289, ... 2 more fields]
+- FileScan csv [cap#94353974,retIC#94353975,resretIC#94353976,numcos#94353977,numdates#94353978,annual_bmret#94353979,annual_ret#94353980,std_ret#94353981,Sharpe_ret#94353982,PctPos_ret#94353983,TR_ret#94353984,IR_ret#94353985,annual_resret#94353986,std_resret#94353987,Sharpe_resret#94353988,PctPos_resret#94353989,TR_resret#94353990,IR_resret#94353991,annual_retnet#94353992,std_retnet#94353993,Sharpe_retnet#94353994,PctPos_retnet#94353995,TR_retnet#94353996,IR_retnet#94353997,... 2 more fields] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/srv/plusamp/data/default/ea-market/output/rankviews_history/rank..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<cap:string,retIC:string,resretIC:string,numcos:string,numdates:string,annual_bmret:string,...
,None)
(11) Filter [codegen id : 1]
Input [2]: [annual_ret#94354242, cap#94354236]
Condition : (isnotnull(cap#94354236) AND (cap#94354236 = 3.0))
(12) Project [codegen id : 1]
Output [1]: [annual_ret#94354242 AS ret_small#94356807]
Input [2]: [annual_ret#94354242, cap#94354236]
(13) BroadcastExchange
Input [1]: [ret_small#94356807]
Arguments: IdentityBroadcastMode, [id=#7533728]
(14) BroadcastNestedLoopJoin [codegen id : 2]
Join condition: None
(15) Project [codegen id : 2]
Output [3]: [ret_large#94356777, ret_small#94356807, (ret_large#94356777 - ret_small#94356807) AS differential#94356893]
Input [2]: [ret_large#94356777, ret_small#94356807]
(16) ColumnarToRow [codegen id : 1]
Input [3]: [ret_large#94356777, ret_small#94356807, differential#94356893]
(17) CollectLimit
Input [3]: [ret_large#94356777, ret_small#94356807, differential#94356893]
Arguments: 10000