Filtering Vs Enriching Data in Apache Spark - Capital One Tech - Medium
I know it’s a vague heading… In the broader scheme of big data processing, there are various options/choices available, with various tradeoffs, on how data can be handled. The intent of this blog is to address that vagueness by comparing two options my team tried out, and why we came to prefer one over the other. Data lies in Capital One’s DNA. All our decisions are data-driven, and being a financial company with a lot of regulatory requirements we are obligated to provide more granular proce...
Filtering Vs Enriching Data in Apache Spark - Capital One Tech - Medium
I know it’s a vague heading… In the broader scheme of big data processing, there are various options/choices available, with various tradeoffs, on how data can be handled. The intent of this blog is to address that vagueness by comparing two options my team tried out, and why we came to prefer one over the other. Data lies in Capital One’s DNA. All our decisions are data-driven, and being a financial company with a lot of regulatory requirements we are obligated to provide more granular proce...
