Category Archives: statistics

Five Thirty Eight podcast: `Can Statistics solve gerrymandering?`

Great podcast, featuring Professor and geometer Moon Duchin, Nate Silver, and Galen Druke. If the link doesn’t work, listen from here or below: Professor Duchin has written extensively on this: M. Duchin, B. E. Tenner, “Discrete geometry for electoral geography”, … Continue reading

Posted in FiveThirtyEight, Nate Silver, point pattern analysis, politics, statistics | Leave a comment

On bag bans and sampling plans

Plastic bag bans are all the rage. It’s not the purpose of this post to take a position on the matter. Before you do, however, I’d recommend checking out this: and especially this: and the Woods Hole Oceanographic Institution has … Continue reading

Posted in bag bans, citizen data, citizen science, Commonwealth of Massachusetts, Ecology Action, evidence, Google, Google Earth, Google Maps, goverance, lifestyle changes, microplastics, municipal solid waste, oceans, open data, planning, plastics, politics, pollution, public health, quantitative ecology, R, R statistical programming language, reasonableness, recycling, rhetorical statistics, sampling, sampling networks, statistics, surveys, sustainability | 1 Comment

Repeating Bullshit

Originally posted on Open Mind:
Question: How does a dumb claim go from just a dumb claim, to accepted canon by the climate change denialati? Answer: Repetition. Yes, keep repeating it. If it’s contradicted by evidence, ignore that or insult…

Posted in American Statistical Association, anomaly detection, changepoint detection, climate change, Grant Foster, Mathematics and Climate Research Network, maths, science, statistics, Tamino, time series, unreason | Leave a comment

A look at an electricity consumption series using SNCDs for clustering

(Slightly amended with code and data link, 12th January 2019.) Prediction of electrical load demand or, in other words, electrical energy consumption is important for the proper operation of electrical grids, at all scales. RTOs and ISOs forecast demand based … Continue reading

Posted in American Statistical Association, consumption, data streams, decentralized electric power generation, dendrogram, divergence measures, efficiency, electricity, electricity markets, energy efficiency, energy utilities, ensembles, evidence, forecasting, grid defection, hierarchical clustering, hydrology, ILSR, information theoretic statistics, local self reliance, Massachusetts, microgrids, NCD, normalized compression divergence, numerical software, open data, prediction, rate of return regulation, Sankey diagram, SNCD, statistical dependence, statistical series, statistics, sustainability, symmetric normalized compression divergence, time series | 1 Comment

Series, symmetrized Normalized Compressed Divergences and their logit transforms

(Major update on 11th January 2019. Minor update on 16th January 2019.) On comparing things The idea of a calculating a distance between series for various purposes has received scholarly attention for quite some time. The most common application is … Continue reading

Posted in Akaike Information Criterion, bridge to somewhere, computation, content-free inference, data science, descriptive statistics, divergence measures, engineering, George Sughihara, information theoretic statistics, likelihood-free, machine learning, mathematics, model comparison, model-free forecasting, multivariate statistics, non-mechanistic modeling, non-parametric statistics, numerical algorithms, statistics, theoretical physics, thermodynamics, time series | 4 Comments

Why Americans and Britons work such long hours

Why Americans and Britons work such long hours.

Posted in business, economics, labor, statistics | Leave a comment

667-per-cm.net, the Podcast: Episode 2, or Probability is Real.

This is the second installment of the Podcast here, hopefully with better sound quality.

Posted in probability, random walks, statistics | Leave a comment

`significance testing`

Image | Posted on by | Leave a comment

Sampling: Rejection, Reservoir, and Slice

An article by Suilou Huang for catatrophe modeler AIR-WorldWide of Boston about rejection sampling in CAT modeling got me thinking about pulling together some notes about sampling algorithms of various kinds. There are, of course, books written about this subject, … Continue reading

Posted in accept-reject methods, American Statistical Association, Bayesian computational methods, catastrophe modeling, data science, diffusion processes, empirical likelihood, Gibbs Sampling, insurance, Markov Chain Monte Carlo, mathematics, Mathematics and Climate Research Network, maths, Monte Carlo Statistical Methods, multivariate statistics, numerical algorithms, numerical analysis, numerical software, numerics, percolation theory, Python 3 programming language, R statistical programming language, Radford Neal, sampling, slice sampling, spatial statistics, statistics, stochastic algorithms, stochastic search | Leave a comment

A quick note on modeling operational risk from count data

The blog statcompute recently featured a proposal encouraging the use of ordinal models for difficult risk regressions involving count data. This is actually a second installment of a two-part post on this problem, the first dealing with flexibility in count … Continue reading

Posted in American Statistical Association, Bayesian, Bayesian computational methods, count data regression, dichotomising continuous variables, dynamic generalized linear models, Frank Harrell, Frequentist, Generalize Additive Models, generalized linear mixed models, generalized linear models, GLMMs, GLMs, John Kruschke, maximum likelihood, model comparison, Monte Carlo Statistical Methods, multivariate statistics, nonlinear, numerical software, numerics, premature categorization, probit regression, statistical regression, statistics | Tagged , , , | Leave a comment

“All of Monsanto’s problems just landed on Bayer” (by Chris Hughes at Bloomberg)

See Chris Hughes’ article. Monsanto has touted Roundup (also known as Glyphosate but more properly as ) as a safe remedy for weed control, often in the taming of so-called “invasive species”. It’s used on playfields where children are exposed … Continue reading

Posted in agroecology, an uncaring American public, business, corporate responsibility, ecology, Ecology Action, environment, environmental law, epidemiology, evidence, invasive species, open data, Peter del Tredici, quantitative biology, quantitative ecology, rights of the inhabitants of the Commonwealth, risk, statistics, sustainability, sustainable landscaping, the right to know, Uncategorized, unreason, Westwood | Leave a comment

Less evidence for a global warming hiatus, and urging more use of Bayesian model averaging in climate science

(This post has been significantly updated midday 15th February 2018.) I’ve written about the supposed global warming hiatus of 2001-2014 before: “‘Overestimated global warming over the past 20 years’ (Fyfe, Gillett, Zwiers, 2013)”, 28 August 2013 “Warming Slowdown?”, Azimuth, Part … Continue reading

Posted in American Statistical Association, Andrew Parnell, anomaly detection, Anthropocene, Bayesian, Bayesian model averaging, Berkeley Earth Surface Temperature project, BEST, climate change, David Spiegelhalter, dependent data, Dublin, GISTEMP, global warming, Grant Foster, HadCRUT4, hiatus, Hyper Anthropocene, JAGS, Markov Chain Monte Carlo, Martyn Plummer, Mathematics and Climate Research Network, MCMC, model-free forecasting, Niamh Cahill, Significance, statistics, Stefan Rahmstorf, Tamino | 2 Comments

Senn’s `… never having to say you are certain’ guest post from Mayo’s blog

via S. Senn: Being a statistician means never having to say you are certain (Guest Post) See also: E. Cai’s blog post “Applied Statistics Lesson of the Day – The Matched Pairs Experimental Design”, from February 2014 A. Deaton, N. … Continue reading

Posted in abstraction, American Association for the Advancement of Science, American Statistical Association, cancer research, data science, ecology, experimental design, generalized linear mixed models, generalized linear models, Mathematics and Climate Research Network, medicine, sampling, statistics, the right to know | Leave a comment

(thought of the day)

One accurate measurement is worth a thousand expert opinions. — Grace Murray Hopper Hat tip to Pat’s blog.

Posted in statistics, Uncategorized | Leave a comment

perceptions of likelihood

That’s from this Github repository, maintained by Zoni Nation, having this description. The original data are from a study by Sherman Kent at the U.S. CIA, and is quoted in at least once outside source discussing the problem. In addition … Continue reading

Posted in anti-intellectualism, Bayes, Bayesian, economics, fear uncertainty and doubt, games of chance, reason, risk, secularism, statistics, the right to be and act stupid, the right to know, the tragedy of our present civilization, unreason | Tagged | Leave a comment

What are the odds of net zero?

What’s the Question? A question was posed by a colleague a couple of months ago: What are the odds of a stock closing at the same price it opened? I found the question interesting, because, at first, it appeared to … Continue reading

Posted in dependent data, evidence, financial series, investing, investments, model-free forecasting, numerical algorithms, state-space models, statistics, time series, trading | Leave a comment

From Xian’s blog, “drivers are not interested in maths formulas”

via drivers are not interested in maths formulas

Posted in Christian Robert, risk, statistics, Uncategorized | Tagged , | Leave a comment

Confidence intervals and that IPCC: Why climate scientists need statistical help

At Andrew Gelman’s blog (Statistical Modeling, Causal Inference, and Social Science), Ben Goodrich makes the interesting observation in a length discussion about confidence intervals, how they should be interpreted, whether or not they have any socially redeeming value, und so … Continue reading

Posted in Bayesian, climate, IPCC, statistics | Leave a comment

Disaster planning in a new climate, inland from the coasts

See Glynis Board’s “The New Normal: Super Storms Highlight Importance Of Disaster Planning”.

Posted in adaptation, American Meteorological Association, American Statistical Association, AMETSOC, Anthropocene, bridge to nowhere, climate disruption, flooding, floods, games of chance, hurricanes, hydrology, meteorology, National Center for Atmospheric Research, NCAR, New England, nor'easters, precipitation, resiliency, risk, statistics, sustainability, the tragedy of our present civilization, the value of financial assets, tragedy of the horizon | Leave a comment

`Insurance companies should collect a carbon levy`

From Anthony J Webster and Richard H Clarke in Nature, “Insurance companies should collect a carbon levy”: Governments juggle too many interests to drive global action on climate change. But the insurance industry is ideally placed. With annual premiums amounting … Continue reading

Posted in American Statistical Association, Anthropocene, attribution, Bloomberg New Energy Finance, bridge to somewhere, business, capitalism, Carbon Tax, climate business, climate economics, corporate litigation on damage from fossil fuel emissions, corporations, Cult of Carbon, economics, energy levy, finance, fossil fuel divestment, fossil fuels, global warming, Hyper Anthropocene, insurance, investments, Joseph Schumpeter, leaving fossil fuels in the ground, moral leadership, statistics, stranded assets, sustainability, the right to know, the value of financial assets, tragedy of the horizon, transparency, zero carbon | Leave a comment

A “capacity for sustained muddle-headedness”

Hat tip to Paul Lauenstein, and his physician brother, suggesting the great insights of the late Dr Larry Weed: Great lines, great quotes, a lot of humor: “… a tolerance of ambiguity …” “Y’know, Pavlov said you must teach a … Continue reading

Posted in American Association for the Advancement of Science, American Statistical Association, anemic data, Bayesian, cardiovascular system, David Spiegelhalter, machine learning, Massachusetts Institute of Technology, medicine, Paul Lauenstein, rationality, reason, reasonableness, risk, statistics | Leave a comment

`Exxon Shareholders Approve Climate Resolution: 62% Vote for Disclosure’

Flash from InsideClimate News: ExxonMobil shareholders voted Wednesday to require the world’s largest oil and gas company to report on the impacts of climate change to its business—defying management, and marking a milestone in a 28-year effort by activist investors. … Continue reading

Posted in Anthropocene, Bloomberg, Bloomberg New Energy Finance, BNEF, bridge to nowhere, business, capitalism, Carbon Worshipers, clean disruption, climate, climate business, climate change, climate disruption, climate economics, corporations, destructive economic development, environmental law, extended supply chains, Exxon, fossil fuel divestment, fossil fuel infrastructure, fossil fuels, Global Carbon Project, global warming, greenhouse gases, Hyper Anthropocene, investing, investments, Joseph Schumpeter, leaving fossil fuels in the ground, making money, Our Children's Trust, petroleum, pollution, rationality, reason, reasonableness, statistics, stranded assets, sustainability, the right to know, the tragedy of our present civilization, the value of financial assets, tragedy of the horizon, zero carbon | Leave a comment

Dikran Marsupial’s excellent bit on hypothesis testing applied to climate, or how it should be applied, if at all

Frankly, I wish some geophysicists and climate scientists wrote more as if they thoroughly understood this, let alone deniers to try to discredit climate disruption. See “What does statistically significant actually mean?”. Of course, while statistical power of a test … Continue reading

Posted in Anthropocene, anti-science, Bayesian, climate change, climate data, climate disruption, D. K. Marsupial, Frequentist, global warming, hiatus, Hyper Anthropocene, ignorance, John Kruschke, regime shifts, statistics, Student t distribution | Leave a comment

The Rule of 135

From SingingBanana.

Posted in Conway's Game of Life, dynamical systems, finite-state machines, mathematical publishing, mathematics, mathematics education, maths, Patterson's Worm, random walks, state-space models, statistical dependence, statistics | Leave a comment

“Bigger Isn’t Always Better When It Comes to Data”: Barry Nussbaum

The President’s Corner in the May 2017 issue of Amstat News, the monthly newsletter of the American Statistical Association (“ASA”), features the interesting exposition by environmental statistician and President of the ASA, Barry Nussbaum, called “Bigger isn’t always better when … Continue reading

Posted in American Statistical Association, emissions, sampling, sampling without replacement, smoothing, spatial statistics, statistics | Leave a comment

`Evidence of a decline in electricity use by U.S. households’ (Prof Lucas Davis, U.C. Berkeley)

This is from a blog post by Professor Lucas Davis at his blog. In addition to the subject, that’s an interesting way of presenting a change over time I’ll need to think about: It seems the model could be used … Continue reading

Posted in American Solar Energy Society, American Statistical Association, anomaly detection, Bloomberg New Energy Finance, BNEF, bridge to somewhere, convergent cross-mapping, decentralized electric power generation, decentralized energy, demand-side solutions, dependent data, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy utilities, engineering, evidence, green tech, local self reliance, Lucas Davis, marginal energy sources, Massachusetts Clean Energy Center, model-free forecasting, multivariate statistics, public utility commissions, rate of return regulation, statistics, Takens embedding theorem | Leave a comment

Investing, and Sharpe’s inequality

See the statement from Sharpe himself. Hat tip to Matt Levine of Bloomberg.

Posted in investments, statistics | 2 Comments

Liang, information flows, causation, and convergent cross-mapping

Someone recommended the work of Liang recently in connection with causation and attribution studies, and their application to CO2 and climate change. Liang’s work is related to information flows and transfer entropies. As far as I know, the definitive work … Continue reading

Posted in Akaike Information Criterion, American Association for the Advancement of Science, Anthropocene, attribution, carbon dioxide, climate, climate change, climate disruption, complex systems, convergent cross-mapping, ecology, Egbert van Nes, Ethan Deyle, Floris Takens, George Sughihara, global warming, Hao Ye, Hyper Anthropocene, information theoretic statistics, Lenny Smith, model-free forecasting, nonlinear systems, physics, statistics, Takens embedding theorem, theoretical physics, Timothy Lenton, Victor Brovkin | 1 Comment

Just because the data lies sometimes doesn’t mean it’s okay to censor it

Or, there’s no such thing as an outlier … Eli put up a post titled “The Data Lies. The Crisis in Observational Science and the Virtue of Strong Theory” at his lagomorph blog. Think of it: Data lying. Obviously this … Continue reading

Posted in Akaike Information Criterion, American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, Anthropocene, Bayes, Bayesian, climate, climate change, climate models, data science, dynamical systems, ecology, Eli Rabett, environment, Ethan Deyle, George Sughihara, Hao Ye, Hyper Anthropocene, information theoretic statistics, IPCC, Kalman filter, kriging, Lenny Smith, maximum likelihood, model comparison, model-free forecasting, physics, quantitative ecology, random walk processes, random walks, science, smart data, state-space models, statistics, Takens embedding theorem, the right to know, Timothy Lenton, Victor Brovkin | 1 Comment

Is the answer to the democratization of Science doing more Citizen Science?

I have been following, with keen interest, the post and comment thread pertaining to “Democratising science” at the blog I monitor daily, … and Then There’s Physics. I think the core subject being discussed is a little different from my … Continue reading

Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, astronomy, astrophysics, biology, citizen data, citizen science, citizenship, data science, ecology, education, environment, evidence, life purpose, local self reliance, marine biology, mathematics, mathematics education, maths, moral leadership, new forms of scientific peer review, open source scientific software, science, science education, statistics, the green century, the right to know | Leave a comment