Fisher information estimation using neural networks

  • Tran Trong Duy Advanced Institute of Engineering and Technology, University of Engineering and Technology, Vietnam National University, Hanoi
  • Nguyen Van Ly Advanced Institute of Engineering and Technology, University of Engineering and Technology, Vietnam National University, Hanoi
  • Nguyen Linh Trung Advanced Institute of Engineering and Technology, University of Engineering and Technology, Vietnam National University, Hanoi
  • Karim Abed-Meraim PRISME Laboratory University of Orleans
DOI: https://doi.org/10.21553/rev-jec.322

Abstract

In estimation theory, the Fisher information matrix (FIM) is a fundamental concept from which we can infer the well-known Cramér-Rao bound. A closed-form expression of the FIM is often intractable due to the lack or sophistication of statistical models. In this paper, we propose a Fisher Information Neural Estimator (FINE) based on neural networks and a relation between the f-divergence and the Fisher information. The proposed method produces an estimate of the FIM directly from observed data. It does not require knowledge or an estimate of the probability density function (pdf), and is therefore universally applicable. The proposed FINE is applicable for not only deterministic parameters but also random parameters. We show via numerical results that the proposed FINE can provide a highly-accurate FIM estimate with a low-computational complexity. Furthermore, we also propose an accelerated FINE version which can be used for scenarios with a high parameter dimension. Finally, we develop an algorithm to choose an appropriate size of the employed neural networks.
Published
2023-11-15
Issue
Vol. 13, No. 1-2 (Jan-Jun, 2023)
Section
Regular articles
An author's submission implies that the manuscript has not been published previously, and is not currently submitted for publication elsewhere. Submission also implies that the Corresponding Author has consent of all authors (the Authors). Upon acceptance for publication transfer of copyright will be made to the Publisher of REV-JEC, who guarantees that full content of the published article is freely distributed on the Journal's website. The copyright transfer gives the Publisher of REV-JEC full authority to resolve any complaints of misuse or abuse (such as infringement or plagiarism) of the published article. The Authors have the freedom to redistribute and reuse the published article in any medium or format for any purpose, provided the original published article is properly cited. An article submission implies author agreement with this policy.