Canonical polyadic decomposition

Before analysis, we reorganized our measurements into a third- or fourth-order tensor, i.e., a three- or four-dimensional array with axes representing parameters over which the profiling was conducted. Within the CP decomposition model, a tensor is described as a sum of the tensor product ($\otimes$) of rank-one components that represent the contribution of each mode. For instance, in the three-mode case:

where $a_r$, $b_r$, and $c_r$ are the r^th column of the factor matrices $A$, $B$, and $C$, which overall summarize how each pattern is represented across the three dimensions.

While many algorithms exist for deriving CP factorizations, we applied alternating least squares (ALS), wherein each mode is iteratively solved using least squares. As an iterative procedure, ALS must be initialized with a starting estimate of the factorization. The CPD decomposition was initialized using the right-hand $r$ eigenvectors from SVD decomposition of the tensor flattened along each mode.

ALS applies the observation that, given factor matrices for two of the modes (e.g., modes 2 and 3), the optimal factor matrix for the remaining mode can be solved for as the least squares solution between the tensor unfolding and Khatri-Rao product of the known factors:

X₍₁₎ represents the tensor unfolding of X along mode 1, and C⊙B represents the Khatri-Rao product of $C$ and $B$. After solving for $A$, ALS proceeds for each of the remaining modes, completing one iteration of the algorithm by building a representation of the other factors using the Khatri-Rao product, and then applying least squares to solve for the left-out factor matrix.

In addition to the ALS scheme described above, we applied the line search routine described by Bro⁵³. Briefly, after two rounds of ALS, the difference between the last two fitting iterations was used to look ahead by a line search step equal to $\sqrt[l]{N}$, where $N$ is the iteration number and $l$ is the line search quotient, initially equal to 2. If the error of the line search-updated factorization is less than the normal ALS update, then the line search result was accepted. Otherwise, the ALS result was used. After four straight rejections the line search quotient was increased by 1 to reduce the line search distance.