This project has moved. For the latest updates, please go here.

SVD problem with LeftTransformMatrix

Nov 14, 2014 at 2:13 PM
Hi,

while using SVD (metanumerics 2.2.0) for a linear regression Y=mX+q -> [Xi,1][m,q]=[Yi] I got a problem with the SVD of the [A]=[Xi,1] matrix. Considering the definition:
'SVD = U∙D∙Vᵀ = A[n,m]
'p=min(m,n)
'D[p,p] = Diagonal matrix
'U[n,p] = Left Orthogonal TransformMatrix
'V[m,p] = Right Orthogonal TransformMatrix

unfortunately the U array retrived by the .LeftTransformMatrix has dimension U[n,n] instead of U[n,p], I wrote the following code for your debug:
 Private Sub DebugSVD()
   Dim A(,) As Double = {{-5.41, 1.0}, {-5.36, 1.0}, {-5.309, 1.0}, {-5.26, 1.0}, _
                         {-5.209, 1.0}, {-5.161, 1.0}, {-5.104, 1.0}, {-5.056, 1.0}, _
                         {-5.01, 1.0}, {-4.96, 1.0}, {-4.903, 1.0}, {-4.858, 1.0}}
   Dim U(,), V(,), Tmp(,) As Double
   Dim n, m, p As Integer
   Dim ASVD As SingularValueDecomposition
  '--------------------------------------

   ASVD = New RectangularMatrix(A).SingularValueDecomposition
  'SVD = U∙D∙Vᵀ = A[n,m]
  'p=min(m,n)
  'D[p,p] = Diagonal matrix 
  'U[n,p] = Left  Orthogonal TransformMatrix
  'V[m,p] = Right Orthogonal TransformMatrix

   U = ASVD.LeftTransformMatrix.ToArray
   V = ASVD.RightTransformMatrix.ToArray

  'BugFixing dimensions SingularValueDecomposition LeftTransformMatrix
   n = UBound(A, 1)   '11 base(0) length=12
   m = UBound(A, 2)   '1  base(0) length=2
   p = Math.Min(m, n) '1  base(0)

   Tmp = ASVD.LeftTransformMatrix.ToArray
   ReDim Preserve Tmp(UBound(Tmp, 1), p)  'remove extra columns
   U = New RectangularMatrix(Tmp).ToArray 'Correct LeftTransformMatrix U[n,p]

ExitDebugSVD:

 End Sub
the result of Tmp = ASVD.LeftTransformMatrix.ToArray is 12x12 matrix not 12x2:
  • Tmp {Length=144} Double(,)
    (0,0)   -0.30350401521306819    Double
    (0,1)   -0.45000785885867128    Double
    (0,2)   -0.31942223577691709    Double
    (0,3)   -0.307218932059838  Double
    (0,4)   -0.29451753431349054    Double
    (0,5)   -0.28256327761104583    Double
    (0,6)   -0.26836759777689279    Double
    (0,7)   -0.25641334107444808    Double
    (0,8)   -0.24495717840127187    Double
    (0,9)   -0.23250482766955866    Double
    (0,10)  -0.21830914783540553    Double
    (0,11)  -0.20710203217686354    Double
    (1,0)   -0.30079637742449877    Double
    (1,1)   -0.36698069433895059    Double
    (1,2)   -0.25283327066281441    Double
    (1,3)   -0.16960046673383344    Double
    (1,4)   -0.082970405501630512   Double
    (1,5)   -0.0014362302242623415  Double
    (1,6)   0.095385602917611945    Double
    (1,7)   0.17691977819497914 Double
    (1,8)   0.25505669616912446 Double
    (1,9)   0.33998812874971551 Double
    (1,10)  0.43680996189159071 Double
    (1,11)  0.51324825121412332 Double
    (2,0)   -0.29803458688015771    Double
    (2,1)   -0.2822929865288431 Double
    (2,2)   0.90152609164245023 Double
    (2,3)   -0.08427979578187661    Double
    (2,4)   -0.0695063316725028 Double
    (2,5)   -0.0556018948636803 Double
    (2,6)   -0.039090376153203661   Double
    (2,7)   -0.025185939344381292   Double
    (2,8)   -0.011860854069259602   Double
    (2,9)   0.0026229342732637176   Double
    (2,10)  0.019134452983740509    Double
    (2,11)  0.032169862492011626    Double
    (3,0)   -0.2953811018473595 Double
    (3,1)   -0.20092636529952446    Double
    (3,2)   -0.0863974769811132 Double
    (3,3)   0.92264996106503028 Double
    (3,4)   -0.067933317703269464   Double
    (3,5)   -0.0590705212499045 Double
    (3,6)   -0.048545950461533628   Double
    (3,7)   -0.039683154008168728   Double
    (3,8)   -0.031189640740360552   Double
    (3,9)   -0.021957561101438768   Double
    (3,10)  -0.011432990313067819   Double
    (3,11)  -0.0031241186380381383  Double
    (4,0)   -0.29261931130301849    Double
    (4,1)   -0.11623865748941897    Double
    (4,2)   -0.073828130038291764   Double
    (4,3)   -0.070137434869821824   Double
    (4,4)   0.93370390091776934 Double
    (4,5)   -0.06268072422332141    Double
    (4,6)   -0.058387466578366645   Double
    (4,7)   -0.054772091719457364   Double
    (4,8)   -0.051307357479669274   Double
    (4,9)   -0.047541342001638766   Double
    (4,10)  -0.043248084356683959   Double
    (4,11)  -0.039858670426456487   Double
    (5,0)   -0.29001997902599175    Double
    (5,1)   -0.036532579550495471   Double
    (5,2)   -0.061998156445047962   Double
    (5,3)   -0.063349101632035554   Double
    (5,4)   -0.0647551874389001 Double
    (5,5)   0.93392143768405089 Double
    (5,6)   -0.067650069982444883   Double
    (5,7)   -0.06897344485949386    Double
    (5,8)   -0.0702416791166659 Double
    (5,9)   -0.071620194613591939   Double
    (5,10)  -0.073191702280087687   Double
    (5,11)  -0.074432366227321148   Double
    (6,0)   -0.28693327194702234    Double
    (6,1)   0.058118388001975781    Double
    (6,2)   -0.047950062803071007   Double
    (6,3)   -0.055287955912164385   Double
    (6,4)   -0.062925354862445035   Double
    (6,5)   -0.07011349505094458    Double
    (6,6)   0.92135058847521223 Double
    (6,7)   -0.085837551713287136   Double
    (6,8)   -0.092726186060599292   Double
    (6,9)   -0.1002138320902862 Double
    (6,10)  -0.10874974856412944    Double
    (6,11)  -0.11548862999084775    Double
    (7,0)   -0.28433393966999565    Double
    (7,1)   0.13782446594089828 Double
    (7,2)   -0.036120089209827337   Double
    (7,3)   -0.048499622674378171   Double
    (7,4)   -0.061384443219114446   Double
    (7,5)   -0.073511333143572263   Double
    (7,6)   -0.087912014928865842   Double
    (7,7)   0.89996109514667655 Double
    (7,8)   -0.11166050769759567    Double
    (7,9)   -0.12429268470223907    Double
    (7,10)  -0.13869336648753278    Double
    (7,11)  -0.15006232579171197    Double
    (8,0)   -0.28184291290451158    Double
    (8,1)   0.21420945729903415 Double
    (8,2)   -0.024783031182968582   Double
    (8,3)   -0.041994136654832925   Double
    (8,4)   -0.059907736227589284   Double
    (8,5)   -0.076767594649007218   Double
    (8,6)   -0.096788676524440911   Double
    (8,7)   -0.1136485349458586 Double
    (8,8)   0.87019410073361569 Double
    (8,9)   -0.14736825178869445    Double
    (8,10)  -0.16738933366412834    Double
    (8,11)  -0.18319545093420764    Double
    (9,0)   -0.27913527511594205    Double
    (9,1)   0.29723662181874533 Double
    (9,2)   -0.012460142023339726   Double
    (9,3)   -0.034922956198805623   Double
    (9,4)   -0.058302619932453247   Double
    (9,5)   -0.080307009328827728   Double
    (9,6)   -0.1064372217370223 Double
    (9,7)   -0.12844161113339647    Double
    (9,8)   -0.14952915097158892    Double
    (9,9)   0.82754961007385464 Double
    (9,10)  -0.19858060233434022    Double
    (9,11)  -0.21920971739344131    Double
    (10,0)  -0.27604856803697264    Double
    (10,1)  0.39188758937121743 Double
    (10,2)  0.0015879516186373538   Double
    (10,3)  -0.026861810478934378   Double
    (10,4)  -0.056472787355998151   Double
    (10,5)  -0.084341942063823186   Double
    (10,6)  -0.11743656327936525    Double
    (10,7)  -0.14530571798718989    Double
    (10,8)  -0.17201365791552253    Double
    (10,9)  -0.20104402740283994    Double
    (10,10) 0.76586135138161771 Double
    (10,11) -0.26026598115696825    Double
    (11,0)  -0.27361169402726004    Double
    (11,1)  0.46661203743895824 Double
    (11,2)  0.012678551862303419    Double
    (11,3)  -0.020497748068509729   Double
    (11,4)  -0.055028182690375717   Double
    (11,5)  -0.087527415275661685   Double
    (11,6)  -0.1261202539706886 Double
    (11,7)  -0.1586194865559741 Double
    (11,8)  -0.18976458445020683    Double
    (11,9)  -0.22361795172654603    Double
    (11,10) -0.2622107904215733 Double
    (11,11) 0.70732117902972114 Double
    
Coordinator
Feb 9, 2015 at 1:19 AM
Edited Feb 9, 2015 at 1:22 AM
You are thinking U has the wrong dimensions because you are thinking of the reduced SVD, not the full SVD. Meta.Numerics produces the full SVD.

Given an M x N matrix A, the full SVD produces A = U * S * V^T where U is M x M and orthogonal, V is N x N and orthogonal, and S is M x N and pseudo-diagonal (the upper N rows are diagonal, the remaining rows are 0). Notice that U and V are both square, S is not. This is the decomposition that we produce.

The reduced SVD produces A = U' * S' * V^T where U' is M x N and S is N x N. Basically, this throws away a bunch of columns of U (so U' can no longer be said to be orthogonal) in order to make S square. It's easy to read off the reduced SVD from the full SVD: just ignore columns of U and rows of S beyond the Mth.

Here is some code that fully reconstructs your matrix (as C) from its SVD:
Dim A(,) As Double = {{-5.41, 1.0}, {-5.36, 1.0}, {-5.309, 1.0}, {-5.26, 1.0}, _
                      {-5.209, 1.0}, {-5.161, 1.0}, {-5.104, 1.0}, {-5.056, 1.0}, _
                      {-5.01, 1.0}, {-4.96, 1.0}, {-4.903, 1.0}, {-4.858, 1.0}}

Dim B As RectangularMatrix = New RectangularMatrix(A)

Dim SVD As SingularValueDecomposition = B.SingularValueDecomposition

Dim U As SquareMatrix = SVD.LeftTransformMatrix()
Dim V As SquareMatrix = SVD.RightTransformMatrix()

Dim S As RectangularMatrix = New RectangularMatrix(12, 2)
For i As Integer = 0 To SVD.Dimension - 1
    S(i, i) = SVD.SingularValue(i)
Next

Dim C As RectangularMatrix = U * S * V.Transpose()
By the way, the Wikipedia article on the Singular Value Decomposition discusses the full vs. reduced SVD.
Feb 9, 2015 at 6:33 AM
I thank you for your replay, which of course it's correct, I was mislead by some doc in accord with "numerical recipes" 3rd edition which on par 2.6 pag 65 mention SVD as A(MxN)=U(MxN) W(NxN) V(NxN)^T without specifying any reduced format. So I learned something more thank you again