Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives when interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the key ideas and algorithms of reinforcement learning. Their discussion ranges from the history of the field's intellectual foundations to the most recent developments and applications. The only necessary mathematical background is familiarity with elementary concepts of probability.The book is divided into three parts. Part I defines the reinforcement learning problem in terms of Markov decision processes. Part II provides basic solution methods: dynamic programming, Monte Carlo methods, and temporal-difference learning. Part III presents a unified view of the solution methods and incorporates artificial neural networks, eligibility traces, and planning; the two final chapters present case studies and consider the future of reinforcement learning.

Co-authored by four leading scientists from academia and industry, Numerical Recipes Third Edition starts with basic mathematics and computer science and proceeds to complete, working routines. Widely recognized as the most comprehensive, accessible and practical basis for scientific computing, this new edition incorporates more than 400 Numerical Recipes routines, many of them new or upgraded. The executable C++ code, now printed in color for easy reading, adopts an object-oriented style particularly suited to scientific applications. The whole book is presented in the informal, easy-to-read style that made earlier editions so popular. Please visit www.nr.com or www.cambridge.org/us/numericalrecipes for more details. More information concerning licenses is available at: www.nr.com/licenses New key features:2 new chapters, 25 new sections, 25% longer than Second EditionThorough upgrades throughout the textOver 100 completely new routines and upgrades of many more.New Classification and Inference chapter, including Gaussian mixture models, HMMs, hierarchical clustering, Support Vector MachinesNew Computational Geometry chapter covers KD trees, quad- and octrees, Delaunay triangulation, and algorithms for lines, polygons, triangles, and spheresNew sections include interior point methods for linear programming, Monte Carlo Markov Chains, spectral and pseudospectral methods for PDEs, and many new statistical distributionsAn expanded treatment of ODEs with completely new routinesPlus comprehensive coverage oflinear algebra, interpolation, special functions, random numbers, nonlinear sets of equations, optimization, eigensystems, Fourier methods and wavelets, statistical tests, ODEs and PDEs, integral equations, and inverse theoryAnd much, much more! This book/CD bundle of the greatly expanded third edition of Numerical Recipes now has wider coverage than ever before, many new, expanded and updated sections, and two completely new chapters. Visit the authors' web site for information about electronic subscriptions www.nr.com/aboutNR3book.html

The annual Neural Information Processing Systems (NIPS) conference is the flagship meeting on neural computation. It draws a diverse group of attendees — physicists, neuroscientists, mathematicians, statisticians, and computer scientists. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning and control, emerging technologies, and applications. Only twenty-five percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains the papers presented at the December, 2004 conference, held in Vancouver.

5 STAR DOODY'S REVIEW!
"This is a simply wonderful book that makes accessible in one place all the details of how the neuron and brain work. The writing is clear. The drawings are elegant and educational. The book is a feast for both the eye and mind. The richness, the beauty, and the complexity of neuroscience is all captured in this superb book."—Doody's Review Service Now in resplendent color, the new edition continues to define the latest in the scientific understanding of the brain, the nervous system, and human behavior. Each chapter is thoroughly revised and includes the impact of molecular biology in the mechanisms underlying developmental processes and in the pathogenesis of disease. Important features to this edition include a new chapter - Genes and Behavior; a complete updating of development of the nervous system; the genetic basis of neurological and psychiatric disease; cognitive neuroscience of perception, planning, action, motivation and memory; ion channel mechanisms; and much more. (20011101)

Widely praised for its student-friendly style and exceptional artwork and pedagogy, Neuroscience: Exploring the Brain is a leading undergraduate textbook on the biology of the brain and the systems that underlie behavior. This edition provides increased coverage of taste and smell, circadian rhythms, brain development, and developmental disorders and includes new information on molecular mechanisms and functional brain imaging. Path of Discovery boxes, written by leading researchers, highlight major current discoveries. In addition, readers will be able to assess their knowledge of neuroanatomy with the Illustrated Guide to Human Neuroanatomy, which includes a perforated self-testing workbook.

This edition's robust ancillary package includes a bound-in student CD-ROM, an Instructor's Resource CD-ROM, and resources online.

Fundamental Neuroscience, 3rd Edition introduces graduate and upper-level undergraduate students to the full range of contemporary neuroscience. Addressing instructor and student feedback on the previous edition, all of the chapters are rewritten to make this book more concise and student-friendly than ever before. Each chapter is once again heavily illustrated and provides clinical boxes describing experiments, disorders, and methodological approaches and concepts. A companion web site contains test questions, and an imagebank of the figures for ready use in presentations, slides, and handouts.

Capturing the promise and excitement of this fast-moving field, Fundamental Neuroscience, 3rd Edition is the text that students will be able to reference throughout their neuroscience careers!

New to this edition:
* 30% new material including new chapters on Dendritic Development and Spine Morphogenesis, Chemical Senses, Cerebellum, Eye Movements, Circadian Timing, Sleep and Dreaming, and Consciousness
* Companion website with figures, web links to additional material, and test questions
* Additional text boxes describing key experiments, disorders, methods, and concepts
* Multiple model system coverage beyond rats, mice, and monkeys
* Extensively expanded index for easier referencing

In the 25 years since From Neuron to Brain was first published, the authors' aim has remained constant—to describe how nerve cells go about their business of transmitting signals, how the signals are put together, and how, out of this integration, higher functions emerge. The new Fourth Edition, while maintaining this focus, has been completely reformatted and updated.

The emphasis, as before, is on experiments, and on the way they are carried out. Using a narrative approach, the authors follow a line from the original inception of a new idea to an account of research being done today. The wealth of new facts, techniques, and concepts, however, presented a challenge in keeping the book to a manageable size. Inevitably, the authors have had to delete descriptions of certain classical experiments.

Largely in response to readers' comments, elements of format and presentation have been changed for this new edition—more headings have been introduced, the paragraphs are shorter, and the illustrations, now in full color, have been clarified. As in previous editions, references are cited throughout the text, and appendices are provided to help readers unfamiliar with the nervous system deal with essential facts and definitions.

Intended for use in upper-level undergraduate, graduate, psychology, and medical school Neuroscience courses, From Neuron to Brain will be of interest to anyone, with or without a specialized background in biological sciences, who is curious about the workings of the nervous system. It presents a readable and coherent account of how cellular and molecular approaches can provide insights into the workings of the brain.

It is widely recognized that the neural basis of brain function can be fully understood only by integrating many disciplines at many levels. Studies of synaptic organization are bringing about a quiet revolution in achieving this goal, as documented by this unique book over the past 30 years. In this fifth edition, the results of the mouse and human genome projects are incorporated for the first time. Molecular biologists interested in functional genomics and proteomics of the brain will find answers here to the critical questions: what are the cell and circuit functions of gene products? Also for the first time, the reader is oriented to supporting neuroscience databases. Among the new advances covered are 2-photon confocal laser microscopy of dendrites and dendritic spines, biochemical analyses, and dual patch and multielectrode recordings, applied together with an increasing range of behavioral and gene-targeting methods. Leading experts in the best understood brain regions bring together the molecular, anatomical, functional, and behavioral data in authoritative integrated accounts. The chapters are organized in the same format, covering the neural elements, synaptic connections, basic circuits, physiology, neurotransmitters, neuromodulators, membrane properties, dendritic properties, and with a final section on how the circuits mediate specific behaviors. The uniform framework for each chapter enables the authors to higlight the principles that are common to all regions, as well as the adaptations unique to each, thus serving as a model for understanding the neural basis of behavior.

This book represents the most complete and authoritative description on the fine structure of the nervous system available in a single volume. Beginning with background material on the neuron, the book then examines specific portions of the nerve cell, and of the various supporting cells. Structure is first described in a general fashion, followed by detailed coverage of the fine structure of each component, with full discussion of how the structural features relate to their functions. Extensively revised and rewritten, this book will bring readers up to date with the many important developments that have taken place since publication of the previous edition. It includes over 130 electron micrographs and line drawings, many of which are new to this edition.

with simulations and illustrations by Richard Gray Problem solving is an indispensable part of learning a quantitative science such as neurophysiology. This text for graduate and advanced undergraduate students in neuroscience, physiology, biophysics, and computational neuroscience provides comprehensive, mathematically sophisticated descriptions of modern principles of cellular neurophysiology. It is the only neurophysiology text that gives detailed derivations of equations, worked examples, and homework problem sets (with complete answers).Developed from notes for the course that the authors have taught since 1983, Foundations of Cellular Neurophysiology covers cellular neurophysiology (also some material at the molecular and systems levels) from its physical and mathematical foundations in a way that is far more rigorous than other commonly used texts in this area.

Dendrites form the major receiving part of neurons. It is within these highly complex, branching structures that the real work of the nervous system takes place. The dendrites of neurons receive thousands of synaptic inputs from other neurons. However, dendrites do more than simply collect and funnel these signals to the soma and axon; they shape and integrate the inputs in complex ways. Despite being discovered over a century ago, dendrites received little research attention until the early 1950s. Over the past few years there has been a dramatic explosion of interest in the function of these beautiful structures. Recent new research has developed out understanding of the properties of dendrites, and their role in neuronal function.

The first edition of Dendrites was a landmark in the literature, stimulating and guiding further research. The new edition substantially updates the earlier volume, and includes 5 new chapters and color illustrations. It gathers new information on dendrites into a single volume, with contributions written by leading researchers in the field. It presents a survey of the current state of our knowledge of dendrites, from their morphology and development through to their electrical, chemical, and computational properties. As such it will not only be of interest to researchers and graduate-level students in neuroscience, but will also be useful to researchers in computer science and IT, psychology, physiology, and biophysics.