# Thread: Fréchet Derivative of the determinant and matrix inverse functions

1. ## Fréchet Derivative of the determinant and matrix inverse functions

Hi all,

I'm trying to get to grips with the Frechet derivative, and whilst I think I've got all the fundamental concepts down, I'm having trouble evaluating some of the trickier limits I've come up against.

The two I'm struggling with currently are the further derivatives of the functions f(A)=$\displaystyle A^{-1}$ on invertible matrices and g(A)=det(A) on all matrices.

For the former, I'm trying to find the Taylor series of f(I+H) about the identity matrix I, so I need to find the general form for a derivative and I've evaluated the first derivative of f at A as Df(A)(H)=$\displaystyle -A^{-1}HA^{-1}$, by using the chain rule in composition with 2 other functions j(A)=A(B^-1) and =(B^-1)A, but I'm having trouble evaluating further derivatives: I've spent a lot of time looking at Df(A+K)(H)-Df(A)(H), but to no avail, and not only that but I need to find a general formula for the n-th derivative in order to calculate the taylor series (unless at some point they become 0, but that seems unlikely!); can anyone suggest how I could get my hands on a general formula? (I'd use induction but I have no idea what I'd be hypothesizing, and if i did use it it would be nice to have a reason why I should think that's the right thing to hypothesize!)

For the latter, I want to find the second derivative of the determinant function at I (just the second derivative this time, not all of them!); I've calculated the first derivative at A to be Dg(A)(H)=$\displaystyle Det(A)Tr(A^{-1}H)$ (or just = Tr(H), at I) but once again I can't work out a nice way (or indeed, any way) to evaluate the k->0 limit of Dg(A+K)(H)-Dg(A)(H) and find the second derivative (could I use the product rule on Det and Tr separately? In that case, I could use a hand calculating the derivative of the trace, since I tried that too already!): could any of you exceedingly smart and handsome () people lend a hand?

Many many thanks, I've spent many hours on these two problems and they're getting to be quite an annoyance, so it'd be lovely to get them sorted before the New Year!

2. Higher Fréchet derivatives are a total pain, because they all belong to different spaces. If $\displaystyle f:V\to W$ is a differentiable map from V to W then, for each v in V, Df(v) is a linear map from V to W. Using the notation L(V,W) for the space of all linear maps from V to W, Df is a map from V to L(V,W). But that means that the second derivative of f is a linear map from V to L(V,W). In other words, $\displaystyle D^2(f)\in L(V,L(V,W))$. And so it goes on: $\displaystyle D^3(f)\in L(V,L(V,L(V,W)))$, ... .

The space $\displaystyle L(V,L(V,W))$ can be identified with the space of bilinear maps from $\displaystyle V\times V$ to W, and similarly for higher derivatives. So $\displaystyle D^n(f)$ can be regarded as an n-linear map from $\displaystyle V\times\cdots\times V$ (n factors) to W.

For the map $\displaystyle f(A) = A^{-1}$ on the space $\displaystyle M_n$ of $\displaystyle n\times n$ matrices, $\displaystyle Df(A)(H) = -A^{-1}HA^{-1}$. The second derivative is given (as a bilinear map) by $\displaystyle D^2f(A)(H,K) = A^{-1}HA^{-1}KA^{-1} + A^{-1}KA^{-1}HA^{-1}$. Notice that in the case of a 1×1 matrix ($\displaystyle A = x$, a scalar, in other words), these formulas reduce to the familiar results $\displaystyle \tfrac d{dx}(x^{-1}) = -x^{-2}$ and $\displaystyle \tfrac {d^2}{dx^2}(x^{-1}) = 2x^{-3}$.

The Fréchet derivatives of the determinant function $\displaystyle \Delta(A) = \det(A)$ are just as bad, if not worse. The determinant is a function from $\displaystyle M_n$ to the scalars, $\displaystyle \Delta:M_n\to\mathbb{C}$. So $\displaystyle D\Delta$ is a linear map from $\displaystyle M_n$ to $\displaystyle \mathbb{C}$, given by $\displaystyle D\Delta(A)(H) = \Delta(A)\text{tr}(A^{-1}H)$. The second derivative $\displaystyle D^2\Delta(A)$ is given as a bilinear map from $\displaystyle M_n\times M_n$ to $\displaystyle \mathbb{C}$ by $\displaystyle D^2\Delta(A)(H,K) = \Delta(A)\bigl(\text{tr}(A^{-1}H)\text{tr}(A^{-1}K) - \text{tr}(A^{-1}HA^{-1}K)\bigr)$.

Having said all that, if you're trying to find the Taylor series of f(I+H) about the identity matrix I, you don't want Fréchet derivatives at all. Just use the binomial expansion $\displaystyle (I+H)^{-1} = I - H + H^2 - H^3 + \ldots$ (valid whenever H is small enough in some suitable metric).

Edit. You asked for the formula for the n'th derivative of the function $\displaystyle f(A) = A^{-1}$. My guess is that $\displaystyle D^n(A)(H_1,\ldots,H_n) = (-1)^n\sum_{\sigma\in S_n}A^{-1}H_{\sigma(1)}A^{-1}H_{\sigma(2)}A^{-1}\cdots A^{-1}H_{\sigma(n)}A^{-1}$ (but I haven't tried to prove that).
{"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 1, "mathjax_display_tex": 0, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "url": "http://mathhelpforum.com/differential-geometry/122024-frechet-derivative-determinant-matrix-inverse-functions.html", "fetch_time": 1527346961000000000, "content_mime_type": "text/html", "warc_filename": "crawl-data/CC-MAIN-2018-22/segments/1526794867417.75/warc/CC-MAIN-20180526131802-20180526151802-00277.warc.gz", "warc_record_offset": 172602413, "warc_record_length": 11655, "token_count": 1400, "char_count": 4825, "score": 3.59375, "int_score": 4, "crawl": "CC-MAIN-2018-22", "snapshot_type": "latest", "language": "en", "language_score": 0.9537131786346436, "file_path": "/sailhome/yjruan/project/latent-lm/data/processed/finemath-4plus/finemath-4plus/train-00020-of-00064.parquet"}