MVA, bibliography of the Course given by Jean-Pierre Nadal

Back to the course main page.

This page: bibliography, lecture notes


Short Bibliography

Most, if not all, books mentionned here may be found at the
IHP
(Institut Henri Poincaré, 11 rue Pierre et Marie Curie, 75005 Paris),
and at the RISC
(Relais d'Information sur les Sciences de la Cognition, Ecole Normale Supérieure, 29 rue d'Ulm, 75005 Paris).

Prehistory

Historical Times

Modern Times

Collective volumes, compilations

Introductory books in French

Information Theory

Short reviews and notes on the two main topics of the course

Other courses and documents


Course material specific to the current academic year

Some documents can be uploaded from the course material web page of the Course Theoretical Neuroscience (ENS, Paris) [access with password: the documents with restricted access cannot be distributed or posted on the internet without authorization. If you are a registered student and do not have the password, ask me.]



Academic year 2016-2017

[Lecture, January 12, 2017]

  • Introduction slides (pdf)
    Synaptic Plasticity: slides; papers here

    [Lecture, Thursday 19 January 2017]

  • Hebbian unsupervised learning and neural coding - the Oja model and variants
    An elementary introduction to the Oja model: lecture notes (slides) (pdf)
    Papers by Erkki Oja, in particular:
    - E. Oja, A simplified neuron model as a principal components analyzer, J. Math. Biol. 15 (1982) 267-273
    - E. Oja, Neural networks, principal components and subspaces, Int. J. of Neural Syst. 1 (1989) 61-68
    T. D. Sanger, Optimal unsupervised learning in a single-layer linear feedforward network, Neural Networks 2 (1989) 459-473
    Hertz, Krogh, and Palmer (1991) (see general references above), Chap. 8.2 pp. 199-210

  • Efficient neural coding

    Introduction - Efficient coding hypothesis - The mutual information; infomax
    Slides.
    Papers:
    Efficient coding hypthesis: H. Barlow, Possible principles underlying the transformations of sensory messages",Sensory Communication, pp. 217-234, 1961 (paper here); "Single units and sensation: a neuron doctrine for perceptual psychology?", 1972 (paper here).
    Information theory: see the general references above. Mutual information, capacity: see the discussion in the introduction of the paper by David Haussler and Manfred Opper, "Mutual information, metric entropy and cumulative relative entropy risk", Ann. Statist. Vol 25, Nber 6 pp 2451-2492 (1997) - paper online here.

    Linear Gaussian models - modeling the retina and the first layers of the visual system
    Slides.
    Papers:
    "Infomax": Linsker, R. (1988), "An application of the principle of maximum information preservation to linear systems", NIPS 1988 (paper here)
    Zhaoping Li, works on early visual coding, in particular the review paper: Optimal Sensory Encoding, in: The Handbook of Brain Theory and Neural Networks, page 815-819, The Second Edition. Michael A. Arbib, Editor MIT Press 2002.
    van Hateren, Biol. Cybernetics 1992, "A theory of maximizing sensory information".

    Non linear models - Single cell: optimization of the transfer function / tuning curve.
    Large signal to noise regime : case of additive noise

    Slides
    Papers:
    Efficient coding in fly vision: S. B. Laughlin, Form and function in retinal processing, TINS, Vol. 10, NO. 11 (1987) pp.478-483.
    Lubomir Kostal, Petr Lansky, Jean-Pierre Rospars, "Efficient Olfactory Coding in the Pheromone Receptor Neuron of a Moth", PLoS Comput Biol 4(4): e1000053 (2008) (paper here).

    Non linear models - Single cell: optimization of the transfer function / tuning curve.
    Case of multiplicative noise: Poisson process
    Large signal to noise regime (large times); low signal to noise regime (short times); from short to large times.
    Slides
    Papers:
    R. B. Stein, "The information capacity of nerve cells using a frequency code", Biophysical Journal, 1967 (paper from PubMedCentral)
    N. Brunel & JPN, "Optimal tuning curves for neurons spiking according to a Poisson process in response to a scalar stimulus" in ESANN'1997 proceedings, D Facto pub., pp. 163-168 (paper here).
    M Bethge, D Rotermund and K Pawelzik, "Optimal Short-Term Population Coding: When Fisher Information Fails", 2003 (paper here)


    [Lecture, Thursday 26 January 2017]

    From Infomax to ICA (Redundancy reduction)
    slides
    Papers:
    Barlow H., Sensory mechanisms, the reduction of redundancy, and intelligence. NPL Symposium on the Mechanization of Thought Process. No. 10, pp 535-539, H.M. Stationery Office, London. Paper from H. Barlow website.
    Nadal J.-P. and Parga N., Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer,

    Population coding - Fisher information vs. Shannon information
    slides
    Papers:
    S. Seung and H. Sompolinsky, " Simple models for reading neuronal population codes", PNAS 90 pp:10749-10753, 1993 (here)
    N. Brunel and J.-P. Nadal, "Mutual information, Fisher information and population coding ", Neural Computation, Vol. 10 issue 7 (October 1, 1998) pp. 1731-1757 (here)
    A. P. Georgopoulos, A. B. Schwartz, R. E. Kettner, "Neuronal population coding of movement direction", Science, September 1986 233: 1416-1419, 1986 (paper here)
    A. Schoups, R. Vogels, N. Qian and G. Orban, "Practising orientation identification improves orientation coding in V1 neurons", Nature 412, 549-553 (2 August 2001) (paper here)
    [Lecture Thursday 9 February 2017]

    Decision making
    slides

    [Lecture Thursday 16 February 2017]

    Decision making - Categorical perception
    slides

    Papers:
    (papers on Decision/decoding will be posted here).

    JPN, Information theoretic approach to neural coding and parameter estimation: a perspective, in: "Probabilistic Models of the Brain: Perception and Neural Function" edited by R. Rao, B. Olshausen, and M. Lewicki, MIT Press, 2002, pp 117-134.

  • Supervised learning
  • The Perceptron: capacity, information capacity, algorithms
    slides, part I

    References:
    The Perceptron: F. Rosenblatt, 1958
    Storage capacity: T. M. Cover, 1965
    Information capacity: paper (JPN, in Cellular Automata, dynamical systems and neural networks, E. Goles & S. Martinez editors, Kluwer/Springer 1994, Book series 'Mathematics and its applications' vol. 282, pp.147-166).
    General references on the perceptron (books and chapters in books):
    M. Minsky, S. Papert, Perceptrons (Cambridge, MIT Press, 1969).
    Hertz, Krogh, and Palmer, "Introduction to the Theory of Neural Computation" (Addison-Wesley, 1991 - now from: Perseus Book Group and Westview Press), Chap. 5.

    [Lecture Thursday 23 February 2017]

    slides, part II
    Papers:
    Storage capacity, the statistical physics approach (E. Gardner): papers.
    The Purkinje cell as a Perceptron: slides, and related papers.
    Learning associations: Hebbian learning and the sparse coding limit (the Willshaw model and variants): see here.


    More course material will be posted here.

    [Lecture Thursday 2 March 2017]

  • Recurrent networks and Models of memory: persistent activity and working memory

    Slides

    References:
    D. Amit, Modeling brain function (Cambridge Univ. Press, 1989).
    Papers: see here, in particular:
    The Hopfield model, 1982 (pdf)
    Y. Miyashita and H. S. Chang. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature, 331:68.70, 1988. (pdf)
    N Brunel (2004) Network models of memory, in Methods and models in neurophysics , C. Chow, B. Gutkin, D. Hansel, C. Meunier and J. Dalibard Eds., Elsevier (pdf)
    Neural networks and other topics: Statistical mechanics approach:
    F. Altarelli, R. Monasson, G. Semerjian, F. Zamponi A review of the statistical mechanics approach to random optimization problems", in Handbook of Satisfiability, edited by Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh, IOS Press (2009) (paper here)
    F. Krzakala, M. Mézard, F. Sausset, Y. Sun, L. Zdeborov´, "Statistical physics-based reconstruction in compressed sensing", Phys. Rev. X 2, 021005 (2012) (paper here).
    Spin glasses:
    Stastistical physics approach: Mezard, Marc; Parisi, Giorgio; Virasoro, Miguel Angel (1987), Spin glass theory and beyond, Singapore: World Scientific, ISBN 9971-5-0115-5.
    Rigorous resuts: M. Talagrand, "Mean Field Models for Spin Glasses", 2010 (here)


    Back to top of page