This talk highlights how to leverage the C++ object-oriented programming paradigm through providing a modern, high-level C++ framework to address the challenges in developing the HiCMA++ numerical library. HiCMA++ exploits the data sparsity structure of dense operators by compressing off-diagonal tiles up to an application’s accuracy threshold. Low-rank approximations constitute a major trend in the linear algebra community as they enable not only to tackle the curse of dimensionality for big data applications but also to solve a broad class of traditional HPC large-scale scientific applications. Once compressed, the matrix is represented by a collection of logical tiles that may be stored in dense or low-rank data layout format. Using function overloading and templating, HiCMA++ enables it to support various low-rank matrix operations with dense and/or low-rank operands, multiple precision generation, compression algorithm variants, and code portability via hardware abstraction — all achieved with high productivity in mind.