Category Archives: stochastic search

Sampling: Rejection, Reservoir, and Slice

An article by Suilou Huang for catatrophe modeler AIR-WorldWide of Boston about rejection sampling in CAT modeling got me thinking about pulling together some notes about sampling algorithms of various kinds. There are, of course, books written about this subject, … Continue reading

Posted in accept-reject methods, American Statistical Association, Bayesian computational methods, catastrophe modeling, data science, diffusion processes, empirical likelihood, Gibbs Sampling, insurance, Markov Chain Monte Carlo, mathematics, Mathematics and Climate Research Network, maths, Monte Carlo Statistical Methods, multivariate statistics, numerical algorithms, numerical analysis, numerical software, numerics, percolation theory, Python 3 programming language, R statistical programming language, Radford Neal, sampling, slice sampling, spatial statistics, statistics, stochastic algorithms, stochastic search | Leave a comment

“Stochastic Parameterization: Towards a new view of weather and climate models”

Judith Berner, Ulrich Achatz, Lauriane Batté, Lisa Bengtsson, Alvaro De La Cámara, Hannah M. Christensen, Matteo Colangeli, Danielle R. B. Coleman, Daan Crommelin, Stamen I. Dolaptchiev, Christian L.E. Franzke, Petra Friederichs, Peter Imkeller, Heikki Järvinen, Stephan Juricke, Vassili Kitsios, François … Continue reading

Posted in biology, climate models, complex systems, convergent cross-mapping, data science, dynamical systems, ecology, Ethan Deyle, Floris Takens, George Sughihara, Hao Ye, likelihood-free, Lorenz, mathematics, meteorological models, model-free forecasting, physics, population biology, population dynamics, quantitative biology, quantitative ecology, Scripps Institution of Oceanography, state-space models, statistical dependence, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, time series, Victor Brovkin | 4 Comments

Six cases of models

The previous post included an attempt to explain land surface temperatures as estimated by the BEST project using a dynamic linear model including regressions on both quarterly CO2 concentrations and ocean heat content. The idea was to check the explanatory … Continue reading

Posted in AMETSOC, anemic data, Anthropocene, astrophysics, Bayesian, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, climate, climate change, climate data, climate disruption, climate models, dlm package, dynamic linear models, dynamical systems, environment, fossil fuels, geophysics, Giovanni Petris, global warming, greenhouse gases, Hyper Anthropocene, information theoretic statistics, maths, maximum likelihood, meteorology, model comparison, numerical software, Patrizia Campagnoli, Rauch-Tung-Striebel, Sonia Petrone, state-space models, stochastic algorithms, stochastic search, SVD, time series | 1 Comment

“Grid shading by simulated annealing” [Martyn Plummer]

Source: Grid shading by simulated annealing (or what I did on my holidays), aka “fun with GCHQ job adverts”, by Martyn Plummer, developer of JAGS. Excerpt: I wanted to solve the puzzle but did not want to sit down with … Continue reading

Posted in approximate Bayesian computation, Bayesian, Bayesian inversion, Boltzmann, BUGS, Christian Robert, Gibbs Sampling, JAGS, likelihood-free, Markov Chain Monte Carlo, Martyn Plummer, mathematics, maths, MCMC, Monte Carlo Statistical Methods, optimization, probabilistic programming, SPSA, stochastic algorithms, stochastic search | Leave a comment

high dimension Metropolis-Hastings algorithms

If attempting to simulate from a multivariate standard normal distribution in a large dimension, when starting from the mode of the target, i.e., its mean γ, leaving the mode γis extremely unlikely, given the huge drop between the value of the density at the mode γ and at likely realisations Continue reading

Posted in Bayes, Bayesian, Bayesian inversion, boosting, chance, Christian Robert, computation, ensembles, Gibbs Sampling, James Spall, Jerome Friedman, Markov Chain Monte Carlo, mathematics, maths, MCMC, Monte Carlo Statistical Methods, multivariate statistics, numerical software, numerics, optimization, reasonableness, Robert Schapire, SPSA, state-space models, statistics, stochastic algorithms, stochastic search, stochastics, Yoav Freund | Leave a comment

Generating supports for classification rules in black box regression models

Inspired by the extensive and excellent work in approximate Bayesian computation (see also), especially that done by Professors Christian Robert and colleagues (see also), and Professor Simon Wood (see also), it occurred to me that the complaints regarding lack of … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, Bayesian inversion, generalized linear models, machine learning, numerical analysis, numerical software, probabilistic programming, rationality, reasonableness, state-space models, statistics, stochastic algorithms, stochastic search, stochastics, support of black boxes | Leave a comment

All I do is complain, complain …

I was reviewing a presentation given as part of a short course in the machine learning genre today, and happened across the following two bullets, under the heading “Strictly Stationary Processes”: Predicting a time series is possible if and only … Continue reading

Posted in bifurcations, chaos, citizen science, convergent cross-mapping, dynamic linear models, dynamical systems, engineering, Floris Takens, generalized linear models, geophysics, George Sughihara, ignorance, Lenny Smith, Lorenz, mathematics, maths, meteorology, prediction, probability, rationality, reasonableness, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, the right to know, time series | 1 Comment

reblog: “Tiny Data, Approximate Bayesian Computation and the Socks of Karl Broman”

It’s Rasmus Bååth, in a post and video of which I am very fond: http://www.sumsar.net/blog/2014/10/tiny-data-and-the-socks-of-karl-broman/.

Posted in approximate Bayesian computation, Bayesian, Bayesian inversion, empirical likelihood, evidence, likelihood-free, probability, rationality, reasonableness, statistics, stochastic algorithms, stochastic search | 1 Comment

Southern New England Meteorology Conference, 24th October 2015

I attending the 2015 edition of the Southern New England Meteorology Conference in Milton, MA, near the Blue Hill, and its Blue Hill Climatological Observatory, of which I am a member as we as of the American Meteorological Society. I … Continue reading

Posted in Anthropocene, capricious gods, climate, Dan Satterfield, dynamical systems, ensembles, ENSO, environment, floods, forecasting, geophysics, Hyper Anthropocene, information theoretic statistics, mesh models, meteorology, model comparison, NCAR, NOAA, nor'easters, oceanography, probability, science, spatial statistics, state-space models, statistics, stochastic algorithms, stochastic search, stochastics, time series | 1 Comment

On differential localization of tumors using relative concentrations of ctDNA. Part 1.

Like most mammalian tissue, tumors often produce shards of DNA as a byproduct of cell death and fracture. This circulating tumor DNA is being studied as a means of detecting tumors or their resurgence after treatment. (See also a Q&A … Continue reading

Posted in approximate Bayesian computation, Bayesian, Bayesian inversion, cardiovascular system, diffusion, dynamic linear models, eigenanalysis, engineering, forecasting, mathematics, maths, medicine, networks, prediction, spatial statistics, statistics, stochastic algorithms, stochastic search, wave equations | 3 Comments

On the Climate Club

But if the other advanced nations had a stick — a tariff of 4 percent on the imports from countries not in the “climate club” — the cost-benefit calculation for the United States would flip. Not participating in the club … Continue reading

Posted in citizenship, civilization, climate, climate change, climate disruption, climate education, ecology, economics, education, environment, ethics, geophysics, global warming, humanism, investing, investment in wind and solar energy, IPCC, mathematics, mathematics education, maths, meteorology, NASA, NCAR, NOAA, open data, open source scientific software, politics, rationality, reasonableness, risk, science, science education, sociology, state-space models, statistics, stochastic search, stochastics, sustainability, temporal myopia, time series, transparency, Unitarian Universalism, UU Humanists, wind power, zero carbon | 2 Comments

“A vignette on Metropolis” (Christian Robert)

This is a very welcome addition by a master of Bayesian computation, providing a great, brief answer for many of my colleagues who ask, “What’s this MCMC thing about anyway?”

Posted in Bayes, Bayesian, Gibbs Sampling, JAGS, MCMC, optimization, probabilistic programming, statistics, stochastic algorithms, stochastic search | Leave a comment

Markov Chain Monte Carlo methods and logistic regression

This post could also be subtitled “Residual deviance isn’t the whole story.” My favorite book on logistic regression is by Dr Joseph Hilbe, Logistic Regression Models, CRC Press, 2009, Chapman & Hill. It is a solidly frequentist text, but its … Continue reading

Posted in Bayes, Bayesian, logistic regression, MCMC, notes, R, statistics, stochastic algorithms, stochastic search | 3 Comments

Christian Robert on the amazing Gibbs sampler

Professor Christian Robert remarks on the amazing Gibbs sampler. Implicitly he’s also underscoring the power of properly done Bayesian computational analysis. For here we have a problem with a posterior distribution having two strong modes, so a point estimate, like … Continue reading

Posted in Bayes, Bayesian, BUGS, Gibbs Sampling, JAGS, mathematics, maths, MCMC, probabilistic programming, rationality, statistics, stochastic algorithms, stochastic search | Leave a comment

Christian Robert on Alan Turing

Alan Turing Institute. See Professor Robert’s earlier post on Turing, too.

Posted in Bayes, Bayesian, citizenship, education, ethics, history, humanism, mathematics, maths, politics, rationality, reasonableness, statistics, stochastic algorithms, stochastic search, the right to know, Wordpress | Tagged | Leave a comment

engineering and understanding with stable models

Stable distributions or Lévy -stable models is a class of probability distributions which contains the Gaussian, the Cauchy (or Lorentz), and the Lévy distribution. They are parameterized by an which is . Values of of 1 or less give distributions … Continue reading

Posted in approximate Bayesian computation, Bayesian, citizen science, climate, climate change, climate education, differential equations, diffusion processes, ecology, economics, forecasting, geophysics, information theoretic statistics, IPCC, mathematics, mathematics education, maths, meteorology, model comparison, NOAA, oceanography, physics, rationality, reasonableness, risk, science, science education, stochastic search, the right to know | Leave a comment

On nested equivalence classes of climate models, ordered by computational complexity

I’m digging into the internals of ABC, for professional and scientific reasons. I’ve linked a great tutorial elsewhere, and argued that this framework, advanced by Wood, and Wilkinson (Robert), and Wilkinson (Darren), and Hartig and colleagues, and Robert and colleagues, … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, biology, ecology, environment, forecasting, geophysics, IPCC, mathematics, maths, MCMC, meteorology, NCAR, NOAA, oceanography, optimization, population biology, Principles of Planetary Climate, probabilistic programming, R, science, stochastic algorithms, stochastic search | Leave a comment

struggling with problems already partly solved by others

Climate modelers and models see as their frontier the problem of dealing with spontaneous dynamics in systems such as atmosphere or ocean which are not directly forced by boundary conditions such as radiative forcing due to increased greenhouse gas (“GHG”) … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, biology, climate, climate education, differential equations, ecology, engineering, environment, geophysics, IPCC, mathematics, mathematics education, meteorology, model comparison, NCAR, NOAA, oceanography, physics, population biology, probabilistic programming, rationality, reasonableness, risk, science, science education, statistics, stochastic algorithms, stochastic search | 1 Comment

Bayesian deconvolution of stick lengths

Consider trying to determine the length of a straight stick. Instead of the measurement errors being clustered about zero, suppose the errors are known to be always positive, that is, no measurement ever underestimates the length of the stick. Such … Continue reading

Posted in Bayesian, Gibbs Sampling, JAGS, mathematics, maths, optimization, probabilistic programming, R, statistics, stochastic algorithms, stochastic search | Leave a comment

The dp-means algorithm of Kulis and Jordan in R and Python

dp-means algorithm. Think k-means but with the number of clusters calculated. By John Myles White, in R. (Github link off that page.) By Scott Hendrickson, in Python. (Github link off that page.)

Posted in Bayesian, Gibbs Sampling, JAGS, mathematics, maths, R, statistics, stochastic algorithms, stochastic search | Tagged | Leave a comment

Blind Bayesian recovery of components of residential solid waste tonnage from totals data

This is a sketch of how maths and statistics can do something called blind source separation, meaning to estimate the components of data given only their totals. Here, I use Bayesian techniques for the purpose, sometimes called Bayesian inversion, using … Continue reading

Posted in Bayesian, BUGS, conservation, consumption, engineering, environment, Gibbs Sampling, JAGS, mathematics, maths, MCMC, MSW, politics, probabilistic programming, R, rationality, recycling, statistics, stochastic algorithms, stochastic search | Leave a comment

singingbanana does “The Lorenz Machine”

On the power of mathematics, and why 55:45 versus 50:50 matters.

Posted in Bayesian, engineering, mathematics, maths, rationality, reasonableness, risk, stochastic algorithms, stochastic search | Tagged | Leave a comment

“The joy and martyrdom of trying to be a Bayesian”

Bayesians have all been there. Some of us don’t depend upon producing publications to assure our pay, so we less have the pressure of pleasing peer reviewers. Nonetheless, it’s all reacting to “What the hell are you doing? I don’t … Continue reading

Posted in Bayesian, BUGS, Gibbs Sampling, JAGS, MCMC, optimization, probabilistic programming, R, rationality, reasonableness, risk, SPSA, statistics, stochastic algorithms, stochastic search | Leave a comment

The zero-crossings trick for JAGS: Finding roots stochastically

BUGS has a “zeros trick” (Lund, Jackson, Best, Thomas, Spiegelhalter, 2013, pages 204-206; see also an online illustration) for specifying a new distribution which is not in the standard set. The idea is to couple an invented-for-the-moment Poisson density to … Continue reading

Posted in Bayesian, BUGS, education, forecasting, Gibbs Sampling, JAGS, mathematics, MCMC, probabilistic programming, R, statistics, stochastic search | Tagged , , | 4 Comments

“Double Plus Big Data”

Big Data. All the rage. Why? Apart from distributed software folks strutting their stuff, something which is likely to be fleeting, especially when quantum computing comes around, what does it buy anyone? I can see four possibilities, which I consider … Continue reading

Posted in Bayesian, education, engineering, investing, mathematics, maths, notes, physics, rationality, reasonableness, statistics, stochastic algorithms, stochastic search | Leave a comment

“Bayes’ theorem in the 21st century”

Professor Bradley Efron wrote a piece on “Bayes’ theorem in the 21st century” in Science for 7th June 2013 which, as always, offers his measured approach to the frequentist-Bayesian controversy (see B. Efron, “A 250 year argument: Belief, behavior, and the … Continue reading

Posted in Bayesian, education, maths, optimization, rationality, SPSA, statistics, stochastic algorithms, stochastic search | Leave a comment