Category Archives: probabilistic programming

Cory Lesmeister’s treatment of Simson’s Paradox (at “Fear and Loathing in Data Science”)

(Updated 2016-05-08, to provide reference for plateaus of ML functions in vicinity of MLE.) Simpson’s Paradox is one of those phenomena of data which really give Statistics a substance and a role, beyond the roles it inherits from, say, theoretical … Continue reading

Posted in Akaike Information Criterion, approximate Bayesian computation, Bayes, Bayesian, evidence, Frequentist, games of chance, information theoretic statistics, Kalman filter, likelihood-free, mathematics, maths, maximum likelihood, Monte Carlo Statistical Methods, probabilistic programming, rationality, Rauch-Tung-Striebel, Simpson's Paradox, state-space models, statistical dependence, statistics, stochastics | Leave a comment

“Grid shading by simulated annealing” [Martyn Plummer]

Source: Grid shading by simulated annealing (or what I did on my holidays), aka “fun with GCHQ job adverts”, by Martyn Plummer, developer of JAGS. Excerpt: I wanted to solve the puzzle but did not want to sit down with … Continue reading

Posted in approximate Bayesian computation, Bayesian, Bayesian inversion, Boltzmann, BUGS, Christian Robert, Gibbs Sampling, JAGS, likelihood-free, Markov Chain Monte Carlo, Martyn Plummer, mathematics, maths, MCMC, Monte Carlo Statistical Methods, optimization, probabilistic programming, SPSA, stochastic algorithms, stochastic search | Leave a comment

Generating supports for classification rules in black box regression models

Inspired by the extensive and excellent work in approximate Bayesian computation (see also), especially that done by Professors Christian Robert and colleagues (see also), and Professor Simon Wood (see also), it occurred to me that the complaints regarding lack of … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, Bayesian inversion, generalized linear models, machine learning, numerical analysis, numerical software, probabilistic programming, rationality, reasonableness, state-space models, statistics, stochastic algorithms, stochastic search, stochastics, support of black boxes | Leave a comment

What the future of energy everywhere looks like

What will the energy landscape look like after utility companies are either dead, dying, or revert to a tiny portion of their territory? Silicon Valley CCE Partnership gives us all a clue. It’s been described in the San Francisco Chronicle, … Continue reading

Posted in adaptation, Anthropocene, capricious gods, chance, citizenship, civilization, clean disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, dynamical systems, economics, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, forecasting, fossil fuel divestment, geophysics, global warming, Hyper Anthropocene, investment in wind and solar energy, living shorelines, mesh models, meteorology, microgrids, mitigation, obfuscating data, oceanography, physical materialism, physics, pipelines, planning, politics, prediction, probabilistic programming, public utility commissions, PUCs, quantum, reasonableness, reproducible research, risk, Sankey diagram, science, sea level rise, selfishness, solar energy, solar power, SolarPV.tv, Spaceship Earth, statistics, stochastic algorithms, stochastics, Svante Arrhenius, taxes, temporal myopia, the right to know, the value of financial assets, transparency, UU Humanists, WHOI, wind energy, wind power, zero carbon | Leave a comment

Solar array with cloud predicting technology launched in WA

Australia’s first grid-connected solar power project with cloud predicting technology launched at Karratha Airport, WA, in bid to smooth solar supply. Source: Solar array with cloud predicting technology launched in WA

Posted in adaptation, Anthropocene, carbon dioxide, citizenship, civilization, clean disruption, climate, climate change, climate disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, dynamic linear models, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, forecasting, geophysics, global warming, Hyper Anthropocene, investment in wind and solar energy, Kalman filter, mathematics, maths, meteorology, microgrids, mitigation, NCAR, numerical software, optimization, physics, prediction, probabilistic programming, rationality, reasonableness, risk, science, solar power, stochastics, sustainability, time series | Leave a comment

Thank You

Originally posted on Open Mind:
To all the readers who make this blog worth writing: Thank you. Thank you for sharing my work. One of the things that makes me proud is that often my blog posts are used as…

Posted in astrophysics, citizen science, climate change, climate data, climate disruption, climate education, climate models, differential equations, dynamical systems, ecology, ensembles, forecasting, games of chance, geophysics, global warming, hiatus, Hyper Anthropocene, IPCC, mathematics, mathematics education, maths, meteorology, model comparison, new forms of scientific peer review, open data, open source scientific software, physics, probabilistic programming, probability, rationality, reasonableness, reproducible research, risk, science, science education, spatial statistics, statistics, Tamino, the right to know, time series, transparency | Leave a comment

“Cauchy Distribution: Evil or Angel?” (from Xian)

Cauchy Distribution: Evil or Angel?. From Professor Christian Robert.

Posted in arXiv, Bayes, Bayesian, Cauchy distribution, information theoretic statistics, mathematics, maths, optimization, probabilistic programming, probability, rationality, reasonableness, statistics, stochastic algorithms, stochastics, Student t distribution | Leave a comment

“A vignette on Metropolis” (Christian Robert)

Originally posted on Xi'an's Og:
Over the past week, I wrote a short introduction to the Metropolis-Hastings algorithm, mostly in the style of our Introduction to Monte Carlo with R book, that is, with very little theory and…

Posted in Bayes, Bayesian, Gibbs Sampling, JAGS, MCMC, optimization, probabilistic programming, statistics, stochastic algorithms, stochastic search | Leave a comment

“Unbiased Bayes for Big Data: Path of partial posteriors” (Christian Robert)

Unbiased Bayes for Big Data: Path of partial posteriors.

Posted in approximate Bayesian computation, Bayes, Bayesian, mathematics, maths, MCMC, optimization, probabilistic programming, statistics, stochastic algorithms | Leave a comment

Christian Robert on the amazing Gibbs sampler

Professor Christian Robert remarks on the amazing Gibbs sampler. Implicitly he’s also underscoring the power of properly done Bayesian computational analysis. For here we have a problem with a posterior distribution having two strong modes, so a point estimate, like … Continue reading

Posted in Bayes, Bayesian, BUGS, Gibbs Sampling, JAGS, mathematics, maths, MCMC, probabilistic programming, rationality, statistics, stochastic algorithms, stochastic search | Leave a comment

The designers of our climate

Originally posted on …and Then There's Physics:
Okay, I finally succumbed and actually waded through some of the new paper by Monckton, Soon, Legates & Briggs called Why models run hot: results from an irreducibly simple climate model. I…

Posted in astrophysics, bridge to nowhere, carbon dioxide, carbon dioxide capture, carbon dioxide sequestration, Carbon Tax, Carl Sagan, citizenship, civilization, climate, climate change, climate education, differential equations, ecology, economics, engineering, environment, ethics, forecasting, fossil fuel divestment, geoengineering, geophysics, humanism, IPCC, mathematics, mathematics education, maths, meteorology, methane, NASA, NCAR, Neill deGrasse Tyson, NOAA, oceanography, open data, open source scientific software, physics, politics, population biology, Principles of Planetary Climate, probabilistic programming, R, rationality, reasonableness, reproducible research, risk, science, science education, scientific publishing, sociology, solar power, statistics, testing, the right to know | 1 Comment

Heads-up limit hold’em poker is solved

This is from today’s news in Science. The full citation is: M. Bowling, N. Burch, M. Johanson, O. Tammelin, “Heads-up limit hold’em poker is solved”, Science, 9 January 2015, 347(6218), 145-149, http://dx.doi.org/10.1126/science.1259433. See also a University of Alberta site where … Continue reading

Posted in games of chance, probabilistic programming, risk | Leave a comment

On nested equivalence classes of climate models, ordered by computational complexity

I’m digging into the internals of ABC, for professional and scientific reasons. I’ve linked a great tutorial elsewhere, and argued that this framework, advanced by Wood, and Wilkinson (Robert), and Wilkinson (Darren), and Hartig and colleagues, and Robert and colleagues, … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, biology, ecology, environment, forecasting, geophysics, IPCC, mathematics, maths, MCMC, meteorology, NCAR, NOAA, oceanography, optimization, population biology, Principles of Planetary Climate, probabilistic programming, R, science, stochastic algorithms, stochastic search | Leave a comment

“[W]e want to model the process as we would simulate it.”

Professor Darren Wilkinson offers a pithy insight on how to go about constructing statistical models, notably hierarchical ones: “… we want to model the process as we would simulate it ….” This appears in his blog post One-way ANOVA with … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, biology, ecology, engineering, forecasting, mathematics, mathematics education, maths, model comparison, optimization, population biology, probabilistic programming, rationality, reasonableness, risk, science, science education, sociology, statistics, stochastic algorithms | Tagged | Leave a comment

struggling with problems already partly solved by others

Climate modelers and models see as their frontier the problem of dealing with spontaneous dynamics in systems such as atmosphere or ocean which are not directly forced by boundary conditions such as radiative forcing due to increased greenhouse gas (“GHG”) … Continue reading

Posted in approximate Bayesian computation, Bayes, Bayesian, biology, climate, climate education, differential equations, ecology, engineering, environment, geophysics, IPCC, mathematics, mathematics education, meteorology, model comparison, NCAR, NOAA, oceanography, physics, population biology, probabilistic programming, rationality, reasonableness, risk, science, science education, statistics, stochastic algorithms, stochastic search | 1 Comment

illustrating particle filters and Bayesian fusion using successive location estimates on the unit circle

Introduction Modern treatments of Bayesian integration to obtain posterior densities often use some form of Markov Chain Monte Carlo (“MCMC”), typically Gibbs sampling. Gibbs works well with many Bayesian hierarchical models. The standard problem-solving situation with these is that a … Continue reading

Posted in Bayes, Bayesian, biology, mathematics, maths, population biology, probabilistic programming, R, statistics, stochastic algorithms | 1 Comment

An equation-free introduction to Bayesian inference

By Tomoharu Eguchi from 2008: “An Introduction to Bayesian Statistics Without Using Equations“.

Posted in Bayes, Bayesian, BUGS, JAGS, mathematics, mathematics education, maths, probabilistic programming, rationality, reasonableness, science education, statistics | Leave a comment

example of Bayesian inversion

This is based upon my solution of Exercise 2.3, page 18, R. Christensen, W. Johnson, A. Branscum, T. E. Hanson, Bayesian Ideas and Data Analysis, Chapman & Hall, 2011. The purpose is to show how information latent in a set … Continue reading

Posted in Bayesian, climate education, ecology, environment, forecasting, geophysics, Gibbs Sampling, JAGS, mathematics, maths, MCMC, physics, probabilistic programming, rationality, reasonableness, risk, science, statistics | 1 Comment

Bayesian deconvolution of stick lengths

Consider trying to determine the length of a straight stick. Instead of the measurement errors being clustered about zero, suppose the errors are known to be always positive, that is, no measurement ever underestimates the length of the stick. Such … Continue reading

Posted in Bayesian, Gibbs Sampling, JAGS, mathematics, maths, optimization, probabilistic programming, R, statistics, stochastic algorithms, stochastic search | Leave a comment

Blind Bayesian recovery of components of residential solid waste tonnage from totals data

This is a sketch of how maths and statistics can do something called blind source separation, meaning to estimate the components of data given only their totals. Here, I use Bayesian techniques for the purpose, sometimes called Bayesian inversion, using … Continue reading

Posted in Bayesian, BUGS, conservation, consumption, engineering, environment, Gibbs Sampling, JAGS, mathematics, maths, MCMC, MSW, politics, probabilistic programming, R, rationality, recycling, statistics, stochastic algorithms, stochastic search | Leave a comment

“The joy and martyrdom of trying to be a Bayesian”

Bayesians have all been there. Some of us don’t depend upon producing publications to assure our pay, so we less have the pressure of pleasing peer reviewers. Nonetheless, it’s all reacting to “What the hell are you doing? I don’t … Continue reading

Posted in Bayesian, BUGS, Gibbs Sampling, JAGS, MCMC, optimization, probabilistic programming, R, rationality, reasonableness, risk, SPSA, statistics, stochastic algorithms, stochastic search | Leave a comment

How fast is JAGS?

How fast is JAGS?.

Posted in BUGS, engineering, Gibbs Sampling, JAGS, mathematics, maths, MCMC, probabilistic programming, R, statistics, stochastic algorithms | Leave a comment

Comment on “How urban anonymity disappears when all data is tracked”, an article in the NY Times

The New York Times has an article titled “How urban anonymity disappears when all data is tracked” by Quentin Hardy which appears in its “Bits” section. I just posted a comment on that article, which is reproduced below: I hope … Continue reading

Posted in citizenship, civilization, economics, education, engineering, Internet, investing, obfuscating data, politics, privacy, probabilistic programming, rationality, reasonableness, risk, statistics, stochastic algorithms | Leave a comment

“The most common fallacy in discussing extreme weather events”: Stefan Rahmstorf

The most common fallacy in discussing extreme weather events.

Posted in carbon dioxide, climate, climate education, forecasting, geophysics, mathematics, maths, meteorology, physics, probabilistic programming, reasonableness, risk, science, statistics | Tagged | Leave a comment

Dr David Gallo of WHOI on today’s “Face the Nation” on CBS: MH370

Good to see Dr Dave Gallo speaking about WHOI’s approach to AF447 and its similarity to MH370. Update. 2014-03-26. WHOI is getting ready to deploy their REMUS 6000 systems. Update. 2014-03-28. The Woods Hole Oceanographic Institution has offered its expertise … Continue reading

Posted in engineering, history, meteorology, oceanography, probabilistic programming, WHOI | Tagged , | Leave a comment

The zero-crossings trick for JAGS: Finding roots stochastically

BUGS has a “zeros trick” (Lund, Jackson, Best, Thomas, Spiegelhalter, 2013, pages 204-206; see also an online illustration) for specifying a new distribution which is not in the standard set. The idea is to couple an invented-for-the-moment Poisson density to … Continue reading

Posted in Bayesian, BUGS, education, forecasting, Gibbs Sampling, JAGS, mathematics, MCMC, probabilistic programming, R, statistics, stochastic search | Tagged , , | 4 Comments

“Data-driven science is a failure of imagination” (Petr Keil)

Happened across this today … I could not agree more: “Data-driven science is a failure of imagination” by Petr Keil. I look forward to reading his posts on Bayesian statistics.

Posted in Bayesian, engineering, history, mathematics, maths, MCMC, probabilistic programming, rationality, science | Tagged , | Leave a comment