A Better Robo Advisory

Building a Better Robo Advisor

The more we learned about the current crop of robo advisory firms, the more we realized we could do better. This brief blog post hits the high points of that thinking.

Not Just the Same Robo Advisory Technology

It appears that all major robo advisory companies use 50+ year-old MPT (modern portfolio theory). At Sigma1 we use so-called post-modern portfolio theory (PMPT) that is much more current. At the heart of PMPT is optimizing return versus semivariance. The details are not important to most people, but the takeaway is the PMPT, in theory, allows greater downside risk mitigation and does not penalize portfolios that have sharp upward jumps.

Robo advisors, we infer, must use some sort of Monte Carlo analysis to estimate “poor market condition” returns. We believe we have superior technology in this area too.

Finally, while most robo advisory firms offer tax loss harvesting, we believe we can 1) set up portfolios that do it better, 2) go beyond just tax loss harvesting to achieve greater portfolio tax efficiency.


Capital Allocation

Let’s start with the idea that CAPM (Capital Asset Pricing Model) is incomplete.   Let me prove it in a  few sentences.  Everyone knows that, for investors, “risk-free” rates are always less than borrowing (margin) rates.  Thus the concept of CAL (the capital asset line) is incomplete.  If I had a sketch-pad I’d supply a drawing showing that there are really three parts of the “CAL” curve…

  1. The traditional CAL that extends from Rf to the tangent intercept with the efficient-frontier curve.
  2. CAC (capital-asset curve)
  3. CAML (capital-asset margin line, pronounced “camel”)

Why?  Because the CAML has it’s own tangent point based on the borrower’s marginal rate.  Because the efficient frontier is monotonically-increasing the CAL and CAML points will be separated by a section of the EF curve I call the CAC.

All of this is so obvious, it almost goes without saying.  It is strange, then, that I haven’t seen it pointed out in graduate finance textbooks, or online.  [If you know of a reference, please comment on this post!]  In reality, the CAL only works for an unleveraged portfolio.

CAPM is Incomplete; Warren Buffett Shows How

Higher risk, higher return, right?  Maybe not… at least on a risk-adjusted basis.  Empirical data suggests that high-beta stock and portfolios do not receive commensurate return.  Quite to the contrary, low-beta stocks and portfolios have received greater returns than CAPM predicts.   In other words, low-beta portfolios (value portfolios in many cases) have had higher historical alphas.  Add leverage, and folks like Warren Buffett have produced high long-term returns.

Black Swans and Grey Swans

On the fringe of modern-portfolio theory (MPT) and post-modern portfolio theory (PMPT), live black swans.   Black swans are essentially the most potent of unknown unknowns, also known as “fat tails”.

At the heart of PMPT is what I call “grey swans.”  This is also called “breakdown of covariance estimates” or, in some contexts, financial contagion.  Grey-swan events are much more common, and somewhat more predictable… That is if one is NOT fixated on variance.

Variance is close, semivariance is closer.  I put forth the idea that PMPT overstates its own potential.  Black swans exists, are underestimated, and essentially impossible to predict.  “Grey swans” are, however, within the realm of PMPT.   They can be measured in retrospect and anticipated in part.

Assets are Incorrectly Priced

CAPM showed a better way to price assets and allocate capital.  The principles of semivariance, commingled with CAPM form a better model for asset valuation.  Simply replacing variance with semivariance changes fifty years of stagnant theory.

Mean-return variance is positively correlated with semivariance (mean semi-variance of asset return), but the correlation is far less than 1.   Further, mean variance is most correlated when it matters most; when asset prices drop.  The primary function of diversification and of hedging is to efficiently reduce variance.  Investors and pragmatists note that this principle matters more when assets crash together — when declines are correlated.

The first step in breaking this mold of contagion is examining what matter more: semivariance.   Simply put, investors care much less about compressed upward variance than they do about compressed downward variance.   They care more about semivariance.  And, eventually, they vote with their remaining assets.

A factor in retaining and growing an AUM base is content clients.  The old rules say that the correct answer the a key Wall Street interview question is win big or lose all (of the client’s money).  The new rules say that clients demand a value-add from their adviser/broker/hybrid.  This value add can be supplied, in part, via using the best parts of PMPT.  Namely semivariance.

That is the the end result of the of the success of semivariance.  The invisible hand of Sigma1, and other forward-looking investment companies, is to guide investors to invest money in the way that best meets their needs.  The eventual result is more efficient allocation of capital.  In the beginning these investors win.  In the end, both investors and the economy wins.  This win/win situation is the end goal of Sigma1.



Financial Software Tech

In order to create software that is appealing to the enterprise market today, Sigma1 must create software for five years from now.   In this post I will answer the questions of why and how Sigma1 software intends to achieve this goal.

The goal of Sigma1 HAL0 software is to solve financial asset allocation problems quickly and efficiently.  HALo is portfolio-optimization software that makes use of a variety of proprietary algorithms.  HALo’s algorithms solve difficult portfolio problems quickly on a single-core computer, and much more rapidly with multi-core systems.

Savvy enterprise software buyers want to buy software that runs well on today’s hardware, but will also run on future generations of compute hardware.   I cannot predict all the trends for future hardware advanced, but I can predict one:  more cores.  Cores per “socket” are increasing on a variety of architectures:  Intel x86, AMD x86, ARM, Itanium, and IBM Power7 to name a few.  Even if this trend slows, as some predict, the “many cores” concept is here to stay and progress.

Simply put — Big Iron applications like portfolio-optimization and portfolio-risk management and modelling are archaic and virtually DOA if they cannot benefit from multi-core compute solutions.   This is why HAL0 is designed from day 1 to utilize multi-core (as well as multi-socket) computing hardware.  Multiprocessing is not a bolt-on retrofit, but an intrinsic part of HAL0 portfolio-optimization software.

That’s the why, now the how.  Google likes to use the phrase “map reduce”  while others like the phase embarrassingly parallel.   I like both terms because it can be embarrassing when a programmer discovers that the problems his software was slogging through in series were being solved in parallel by another programmer who mapped them to parallel sub-problems.

The “how” for HAL0’s core algorithm is multi-layered.   Some of these layers are trade secrets, I can disclose one.  Portfolio optimization involves creating an “efficient frontier” comprised of various portfolios along the frontier.  Each of these portfolios can be farmed out in parallel to evaluate its risk and reward values.   Depending on the parameters of a particular portfolio-optimization problem this first-order parallelism can provide roughly a 2-10x speedup — parallel, but not massively parallel.

HALo was developed under a paradigm I call CAP (congruent and parallel).  Congruent in this context means that given the same starting configuration, HAL0 will always produce the same result.  This is generally easy for single-threaded programs to accomplish, but often more difficult for programs running multiple threads on multiple cores.    Maintaining congruence is extremely helpful in debugging parallel software, and is thus very important to Sigma1 software.  [Coherent or Deterministic could be used in lieu of Congruent.]

As HAL0 development continued, I expanded the CAP acronym to CHIRP (Congruent, Heterogeneous, Intrinsically Recursively Parallel).   Not only does CHIRP have a more open, happier connotation that CAP, it adds two additional tenets:  heterogeneity and recursion.

Heterogeneity, in the context of CHIRP, means being able to run, in parallel, on a variety of machines will different computing capabilities.  On on end of the spectrum, rather than requiring all machines in the cloud or compute queue having the exact same specs (CPU frequency, amount of RAM, etc), the machines can be different.  On the other end of the spectrum, heterogeneity means running in parallel on multiple machines with different architectures (say x86 and ARM, or x86 and GPGPUs).  This is not to say that HAL0 has complete heterogeneous support; it does not.  HALo is, however, architected with modest support for heterogeneous solutions and extensibility for future enhancements.

The recursive part of CHIRP is very important.  Recursively parallel means that the same code can be run (forked) to solve sub-problems in parallel, and those sub-problems can be divided into sub-sub problems, etc.   This means that the same tuned, tight, and tested code can leveraged in a massively parallel fashion.

By far the most performance-enhancing piece of HAL0 portfolio-optimization CHIRP is RP.  The RP optimizations are projected to produce speedups of 50 to 100X over single-threaded performance (in a compute environment with, for example, 20 servers with 10 cores each).  Moreover, the RP parts of HAL0 only require moderate bandwidth and are tolerant of relatively high latency (say, 100 ms).

Bottom line:  HAL0 portfolio-optimization software is designed to be scalable and massively parallel.




Building a Financial Software Company


Saturday, I had a fascinating morning starting a 10-week entrepreneurship program.  I sat face-to-face with a couple of people who could likely write 7-figure checks that would clear the bank.

Tuesday was day 2 of the entrepreneurship class.  I can sum up the general mood of the class with two words:  energy and optimism.  Speakers, coaches, and students were increasingly energized as the 3.5 hour class progressed.  There are some talented students in my class who have already have revenue, employees, and venture capital.  Others have a Ph.D. thesis and novel ideas from their post-doc work.  I am presently somewhere in between:  I am beyond the idea phase and well into the software product development phase.  I have demo-able software and am self-funded to the tune of approx. $50,000.

Apparently my ideas and software are starting to get traction within the group.  It is possible that some college students will write the Sigma1 Financial business plan as a for-credit exercise.  If this pans out, I get a free first-draft business plan, and they get college credit.  That spells “win-win” to me.

I have also learned that my “elevator pitch” needs work.  My off-the-cuff pitch on Day 1 was my best.  Successive effort have been worse.  Feedback examples include:  I felt buried; too much financial jargon; too long.  This is great feedback!  Better ideas from the group include gems such as:

  • Financial Risk:  If you can see it, you can avoid it.  Understand risk, visualize risk, and using proprietary software decrease risk by X%.
  • Nobody likes risk, but almost nobody truly understands it.  Sigma1 software does.
  • Helping financial advisers model and explain risk to their clients.
  • Visualizing risk… Really!

Here’s the revised elevator pitch based off of their feedback.   “Hi, I founded Sigma1 Software.   If you own an investment portfolio there is one think you should know:  it was probably constructed with using flawed risk models.   Sigma1 Software can analyze your existing portfolio using better risk models.  It will also build alternative portfolios and display the results visually.  Our software not only reduces risk; it helps you visualize risk.

Few people like financial risk, and fewer still truly understand risk.  Sigma1 Software does, and it uses advanced risk models and proprietary algorithms to help you build a better portfolio.

That’s the pitch:  Smarter risk models, risk reduction, and powerful visualization tools.  It’s what we do at Sigma1 Software.”

Building a Solid Team

To take Sigma1 Software to the next level, I need a great team.  At a minimum I need a salesperson.  This person needs both sales experience and experience in the financial industry.  Having a wide network of connections to financial professionals is a plus.  I’m looking for someone will no only close deals, but work with me and our attorneys to construct standard contracts and pricing models.  A person who will build and maintain long-term relationships with our customers while finding new customers.  Finally a salesperson who listens to customer needs in order to help us improve our software and software support.

Next on my hiring list is a second software developer specializing in user interface development for Windows, Linux, and Web applications. I’m looking for a developer who also has the interpersonal skills to 1) contribute to a highly collaborative environment,  2)  conduct customer training, 3) join sales visits to demo Sigma1 software and answer technical questions.  Pluses would be Windows and/or Linux sysadmin and IT skills.  The software developer position would also require occasional work performing unit, regression, and beta testing of the portfolio-optimization suite.

It is crucial that any employees, partners, or employee/partners believe in Sigma1’s mission:  To build and sell revolutionary portfolio-optimization technologies which will revitalize and reinvent the financial industry.  To that end, equity in the form of voting, non-voting, dilutable, and possibly non-dilutable stock are likely to be an important part of any hiring agreement.

Getting Technical

Software Scalability

Without IPC (Inter-process communication) scalability is virtually impossible.  With IPC comes numerous choices and tradeoffs:  pipes, named pipes, messages, semaphore-managed file-sharing, memory-sharing, sockets, socket pairs… to name some.   The tradeoffs involve platform support, job granularity, and performance.

After much thought, I have temporarily committed to Linux/UNIX named pipes as the primary IPC mechanism.  The pros of this choice include robustness, reasonable speed, and a wide range of support for multi-thread, multi-core, and multi-server parallelism.  The primary downside: lack of Windows compatibility.

The HAL0 portfolio-optimization suite can still run on Windows, but for now Windows parallelism is limited to thread-level scalability on a single machine.


Linux and UNIX programs often communicate very well together with streaming data.  [For this post, I’ll used the term Linux to generally refer to Linux and/or UNIX].  Streaming data has its limitations, however.  Perhaps the biggest limitation is “large-grain granularity”.  By this I mean that an interaction involves loading the program (high-latency), processing the stream, closing the program.  Streaming is expensive for fine-grain ad hoc requests because of the open/close overhead.

Named pipes (especially when properly paired) are an elegant way around the open/close overhead issue.

It is often a simple task to modify a Linux program to support named pipes.  It does not matter if said program is written in C++, Python, Java, Ruby, Perl, shell, etc.  If a program can be modified to accept input from a named pipe and write results to another named pipe, it can be integrated with other program that supports a common name, pipe-based API.  HAL0 financial software does just that.

Scalability Squared

HAL0 software forms the core engine or kernel of the Sigma1 portfolio software.  In parallel mode, HAL0 spawns worker jobs that compute portfolio metrics.  HAL0 gathers, organizes, and evaluates the results.  Then, based on the past and current wave of results, HAL0 identifies the most fruitful next wave of results to explore and “farms out” the next wave of computing.  This is the primary means of achieving scalability:  the worker jobs are distributed to other cores and other other servers.  These “other” compute resources perform the massive heavy-lifting in parallel, speeding up computation immensely.

When there are enough worker jobs, there comes a point when the worker jobs cease to be the primary compute-speed limiter.  This is where Amdahl’s law really kicks in.  At some point maximum speedup is achieved, limited by the “core” processes ability to send, wait for, receive and process worker-job data.

If the “core” (or master or “boss”) process itself can be split into parallel processes, an additional level of scalability kicks in.  This core HAL0 algorithm is designed to do just that.

Based on preliminary estimates, it can scale efficiently to 4 cores, delivering up to a 3.5X core speedup.   Additionally the current HAL0 periphery for each HAL0 instance scales efficiently to up to 10-ways, providing about a 6X additional speedup per instance (depending on IPC latency). Ideally, Sigma1 portfolio-optimization software can provide up to a 21X speedup (3.5 times 6) operating in parallel mode versus single-CPU mode.

There are many caveats in the preceding paragraph.  Right now I am focused on implementing and testing scalability, less so than on optimizing it.  For example I am currently implementing single-kernel instance scalability in a manner than is deterministic, repeatable, and producing results identical to single-CPU operation.  This limits the speedup, but makes regression testing practical.  Regression testing in turn helps keep the code robust and reliable.

Portfolio-Optimization Software for the Enterprise

So far, ensuring that Sigma1 portfolio software is capable of massive scalability has roughly tripled the software development effort. This is obviously slowing time-to-market, but I continue to believe it is worth the effort and schedule impact. First, scalability is a key product differentiator for enterprise-level customers. Second, supporting scalability from day 1 is much easier and more reliable that trying to retrofit scalability into an intrenched software architecture.


Signficance of the Name Sigma 1 (Σ1)

Σ1 is shorthand for Σwi = 1.   This means that any given instant a portfolio can be expressed as a series of investment weights which always sum to 1.  This principle even applies to long-short funds with positive and negative investment weightings and leveraged funds with negative cash weightings.  This is the primary meaning of Sigma1.

σ1, with a lower-case sigma, is shorthand for one standard deviation (σ times 1).  This is the second thing I associate with Sigma1 Software  — a classic measure of financial risk.

While a portfolio manager may have many responsibilities, his or her “job one” is deciding which investments in what proportion.   It is surprising how challenging this job can be, given that even a 100-asset portfolio can have many more than trillions of trillions of possible weightings [actually a gross understatement].

Wrapping up;  Sigma1 is a startup creating financial software focused on investment portfolio optimization.  In this context “investment portfolio” can mean a proprietary-trading fund, a mutual fund, an ETF, a financial endowment, “index plus”,  or other managed investment.  At the end the day, Sigma1 means business.


Benchmarking Financial Algorithms

In my last post I showed that there are far more that a googol permutations of portfolio of 100 assets with (positive, non-zero) weights in increments of 10 basis points, or 0.1%.    That number can be expressed as C(999,99), or C(999,900) or 999!/(99!*900!), or ~6.385*10138.  Out of sheer audacity, I will call this number Balhiser’s first constant (Kβ1).  [Wouldn’t it be ironic and embarrassing if my math was incorrect?]

In the spirit of Alan Turing’s 100th birthday today and David Hilbert’s 23 unsolved problems of 1900, I propose the creation of an initial set of financial problems to rate the general effectiveness of various portfolio-optimization algorithms.  These problems would be of a similar form:  each having a search space of Kβ1. There would be 23 initial problems P1…P23.  Each would have a series of 37 monthly absolute returns.  Each security will have an expected annualized 3-year return (some based on the historic 37-month returns, others independent).  The challenge for any algorithm A to score the best average score on these problems.

I propose the following scoring measures:  1) S”(A) (S double prime) which simply computes the least average semi-variance portfolio independent of expected return.  2) S'(A) which computes the best average semi-variance and expected return efficient frontier versus a baseline frontier.  3) S(A) which computes the best average semi-variance, variance, and expected return efficient frontier surface versus a baseline surface.  Any algorithm would be disqualified if any single test took longer than 10 minutes.  Similarly any algorithm would be disqualified if it failed to produce a “sufficient solution density and breadth” for S’ and S” on any test.  Obviously, a standard benchmark computer would be required.  Any OS, supporting software, etc could be used for purposes of benchmarking.

The benchmark computer would likely be a well-equipped multi-core system such as a 32 GB Intel  i7-3770 system.  There could be separate benchmarks for parallel computing, where the algorithm + hardware was tested as holistic system.

I propose these initial portfolio benchmarks for a variety of reasons.  1)  Similar standardized benchmarks have been very helpful in evaluating and improving algorithms in other fields such as electrical engineering.  2)  Providing a standard that helps separate statistically significant from anecdotal inference. 3)  Illustrate both the challenge and the opportunity for financial algorithms to solve important investing problems. 4)  Lowering barriers to entry for financial algorithm developers (and thus lowering the cost of high-quality algorithms to financial businesses).  5)  I believe HAL0 can provide superior results.

Entrepreneurs: Discontent can be Good

Pros and cons image

Evaluating Pros & Cons of Entrepreneurship

Technologies Mature and Innovator’s Discontent Grows

Discontent, in moderation, can be a powerful motivator for entrepreneurial spirit. I have been continuously employed for 15 years at 3 different Fortune-500 technology companies.  All three companies have excellent benefits such as stock grants, stock options, stock purchase plans, profit sharing and bonuses, 401k matching, quality health insurance, and paid vacations.  These benefits are very tough to walk away from.  In general, I am very content with my benefits and somewhat content with my salary.

My level of satisfaction with the job itself has fluctuated widely over time.  The most satisfying times are when I have made lasting changes to how CPUs are designed.  Most of these improvements are smaller than the round-off error in the multi-billion dollar corporate balance sheet.  However, at least one innovation I worked on, that was part of a small team effort, will likely be large enough to measurably boost the company’s bottom line.  The company was insightful enough to give an award to our team for this accomplishment.

Unfortunately, most of the work my coworkers and I do is simply about getting things done (aka execution).  Much of my work is not technically challenging — the key challenge is managing the shear volume of work.  This is a change from 15 years ago, when there were many new and interesting technical problems to be solved.

Today, we computer design engineers, have become at least 10X more productive than we were 10-15 years ago.  We each deliver millions of transistors in less time than we would have delivered tens or occasionally hundreds of thousands of transistors a decade ago.

Ten years ago I was providing unique contributions to the companies I worked for on a fairly regular basis — about 5-6 times a year.  Today, I am making unique contributions about once per year on average.  This is not due to a lack of ideas; it is simply due to a lack of time.  Unlike the “old days”, I seldom get permission to work on innovation.  And when I do, it is in the realm of “focused innovation”, which is all-too-often an oxymoron.

Discontent Leads to Change, Sometimes to Entrepreneurship

My doctor is a former electrical engineer.  He experienced his company rapidly outsourcing engineering work to India and other low-wage countries, and decided to go to medical school.  He now works for a medical group, and thus I don’t see his career change as entrepreneurial.

My story is different than my doctor’s.  I saw the similar writing on the wall that he saw, but I interpreted it a bit differently.  I saw a future of wage stagnation rather than unemployment.  I saw that I must climb the electrical engineering food chain to what many consider the top — CPU Hardware Design.  And I changed companies to make that climb.  Simultaneously I took grad school classes to boost my knowledge in electrical and computer engineering as well as in finance.

Why study finance? 1) I have passion for finance  2) After studying electrical engineering, the math is easy.  3) Finance is a field that does not compete with my employer.  4)  To understand technology and money is to better understand what “makes the world go ’round.” 5)  To become a better, or at least more knowledgeable, investor.  6) As a backup career option.  7) To become a financial entrepreneur.

Serial Entrepreneur Considers Leaving the Corporate Cocoon

It could be argued that I have been an entrepreneur since I managed a paper route at the age of 12.  I sold subscriptions door-to-door, I collected money for subscriptions, and I delivered papers.  In high school I had did high-tech work for a print and publishing company.  I telecommuted at age 16, using a 2400 baud modem to receive raw data and send back publication-ready charts and graphs.  I even turned the college scholarship effort into an entrepreneurial enterprise:  submitting over 50 applications and being awarded over $100,000 in merit-based scholarships.

I won’t bore you with the details of my intervening entrepreneurial efforts, other than to say that I was trading and auctioning products in 1995 on the internet (and Usenet), the same year that eBay was founded.

My fiancée has been a successful entrepreneur for 3 and 1/2 years.  Her company’s revenue and earnings growth has been explosive (and the trend continues), while my salary growth over 7 years has not even kept pace with inflation.  I have little doubt I could literally walk across the street and get a similar-paying job at 2 different tech companies… tech companies that are not nearly as financially secure as my present company.  I could probably even get a 10%-15% pay bump.  However, I have connections at these companies who, in aggregate, tell me that work there is simply less rewarding relative to the work at my present employer.  Simply put, I work on one of the best projects for one of the best companies in the industry.  My coworkers are likeable and extremely talented.  Why would I consider giving this up?

I can answer this question with a question:  “Do you want to change the world (for the better)?”  My answer is a resounding “YES!”  Do I see that happening in my corporate job?  Sadly, it seems very unlikely, a least not in any profound way.

I am young enough that I can take a multi-year stab at full-time entrepreneurship.  I have saved enough, and positioned myself financially so that I can live for 3-5 years without any earned income.  And, if my venture should not succeed, I believe I can re-enter the corporate electrical engineering, software development, or financial services job markets.

Caution and Calculated Risk in the Mind of an Entrepreneur

My heart wants to put in my two-weeks notice tomorrow and dive straight into making Sigma1 portfolio-optimization software into the premier financial software in the world.  My rational side has a very different perspective;  Sigma1 must generate sufficient revenue (or meet other financial criteria) to justify a radical career change.  Scenario #1 involves achieving revenue equal to my current total compensation for present job… as a strictly self-financed undertaking.  Scenario #2 envisions a large (>$1M) external capital investment out of which I retain significant equity ownership (at least 50.1% non-dilutable) and a 3-year salary contract for $100,000. per year.  Scenario #3 entails a very large external capital investment (>$2.5M), a longer and/or larger salary and benefits guarantee, non-dilutable equity retention of at least 20%, a voting seat on the board for 3+ years, and a 5+ year contact to control software development with the title of CTO.

Obviously, I don’t know what the future holds.  My employer should not have any immediate concerns.  I fully intend to be model employee — out of personal responsibility and because if my ventures don’t work out, I want a chance to return in a few years on good terms.