Category Archives: statistical dependence

Weekend break: Theme for Earth Day

Posted on 6 April 2019 by ecoquant

By John Williams:

Posted in agroecology, Aldo Leopold, American Association for the Advancement of Science, American Statistical Association, an uncaring American public, argoecology, biology, Botany, Buckminster Fuller, climate, David Suzuki, dynamical systems, E. O. Wilson, earth, Earth Day, ecological disruption, ecological services, Ecological Society of America, ecology, Ecology Action, ecomodernism, ecopragmatism, ecopragmatist, Eli Rabett, environment, Equiterre, evolution, fragmentation of ecosystems, global warming, green tech, greenhouse gases, greenwashing, invasive species, investing, investment in wind and solar energy, investments, Lawrence Berkeley National Laboratory, Lotka-Volterra systems, marine biology, Mathematics and Climate Research Network, microbiomes, NOAA, oceans, Peter del Tredici, Peter Diggle, Pharyngula, physical materialism, quantitative biology, quantitative ecology, rate of return regulation, scientific publishing, Spaceship Earth, statistical dependence, Stefan Rahmstorf, Tamino | Leave a comment

Still a climate hawk, and appreciate all my climate friends: To the climate deniers, the greenwashers, the liberal environmental opportunists, and the environmental purists who will never compromise …

Posted on 4 April 2019 by ecoquant

“Not ready to make nice” (Dixie Chicks) I stick by my friends in these hard times: Tamino’s community The Azimuth Project Woods Hole Oceanographic Institution The American Statistical Association The International Society for Bayesian Analysis Losing Earth: The decade we … Continue reading →

Posted in American Association for the Advancement of Science, American Statistical Association, Anthropocene, Bayesian, climate change, climate disruption, climate economics, climate grief, coastal investment risks, ecological disruption, ecological services, ecomodernism, ecopragmatism, engineering, environment, flooding, global warming, Grant Foster, Humans have a lot to answer for, Hyper Anthropocene, investment in wind and solar energy, investments, Joseph Schumpeter, Mathematics and Climate Research Network, mathematics education, personal purity, population biology, quantitative biology, quantitative ecology, regulatory capture, risk, riverine flooding, sampling without replacement, Scituate, secularism, shorelines, solar democracy, solar domination, solar energy, Solar Freakin' Roadways, solar power, SolarPV.tv, Spaceship Earth, statistical dependence, SunPower, the energy of the people, the green century, the tragedy of our present civilization, the value of financial assets, tragedy of the horizon, Unitarian Universalism, unreason, utility company death spiral, UU Needham, Wally Broecker, Walt Disney Company, Woods Hole Oceanographic Institution, ``The tide is risin'/And so are we'' | 1 Comment

Stream flow and P-splines: Using built-in estimates for smoothing

Posted on 28 January 2019 by ecoquant

Mother Brook in Dedham Massachusetts was the first man-made canal in the United States. Dug in 1639, it connects the Charles River at Dedham, to the Neponset River in the Hyde Park section of Boston. It was originally an important … Continue reading →

Posted in American Statistical Association, citizen data, citizen science, Clausius-Clapeyron equation, Commonwealth of Massachusetts, cross-validation, data science, dependent data, descriptive statistics, dynamic linear models, empirical likelihood, environment, flooding, floods, Grant Foster, hydrology, likelihood-free, meteorological models, model-free forecasting, non-mechanistic modeling, non-parametric, non-parametric model, non-parametric statistics, numerical algorithms, precipitation, quantitative ecology, statistical dependence, statistical series, stream flow, Tamino, the bootstrap, time series, water vapor | 2 Comments

A look at an electricity consumption series using SNCDs for clustering

Posted on 11 January 2019 by ecoquant

(Slightly amended with code and data link, 12th January 2019.) Prediction of electrical load demand or, in other words, electrical energy consumption is important for the proper operation of electrical grids, at all scales. RTOs and ISOs forecast demand based … Continue reading →

Posted in American Statistical Association, consumption, data streams, decentralized electric power generation, dendrogram, divergence measures, efficiency, electricity, electricity markets, energy efficiency, energy utilities, ensembles, evidence, forecasting, grid defection, hierarchical clustering, hydrology, ILSR, information theoretic statistics, local self reliance, Massachusetts, microgrids, NCD, normalized compression divergence, numerical software, open data, prediction, rate of return regulation, Sankey diagram, SNCD, statistical dependence, statistical series, statistics, sustainability, symmetric normalized compression divergence, time series | 1 Comment

The Rule of 135

Posted on 29 May 2017 by ecoquant

From SingingBanana.

Posted in Conway's Game of Life, dynamical systems, finite-state machines, mathematical publishing, mathematics, mathematics education, maths, Patterson's Worm, random walks, state-space models, statistical dependence, statistics | Leave a comment

`Letter to Lamar Smith’

Posted on 28 March 2017 by ecoquant

On Ed Hawkins’ blog. The Committee on Science, Space & Technology of the US House of Representatives conducts regular evidence hearings on various science topics. On Wednesday 29th March, there is a hearing on “Climate science: assumptions, policy implications, and … Continue reading →

Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, anemic data, anomaly detection, Anthropocene, Ben Santer, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, changepoint detection, climate, climate change, climate data, climate disruption, Climate Lab Book, climate zombies, dependent data, environment, fossil fuel divestment, geophysics, global warming, greenhouse gases, Humans have a lot to answer for, Hyper Anthropocene, leaving fossil fuels in the ground, meteorology, MIchael Mann, Our Children's Trust, physics, science, smoothing, statistical dependence, the right to be and act stupid, the right to know, the tragedy of our present civilization, the value of financial assets, time series | Leave a comment

Why smooth?

Posted on 8 December 2016 by ecoquant

I’ve encountered a number of blog posts this week which seem not to understand the Bias-Variance Tradeoff in regard to Mean-Squared-Error. These arose in connection with smoothing splines, which I was studying in connection with multivariate adaptive regression splines, that … Continue reading →

Posted in Akaike Information Criterion, American Statistical Association, Antarctica, carbon dioxide, climate change, denial, global warming, information theoretic statistics, likelihood-free, multivariate adaptive regression splines, non-parametric model, science denier, smoothing, splines, statistical dependence | 1 Comment

Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories

Posted on 11 August 2016 by ecoquant

(Slight update, 28th June 2020.) I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random … Continue reading →

Posted in American Meteorological Association, American Statistical Association, AMETSOC, Arthur Charpentier, atmosphere, diffusion, diffusion processes, dynamic linear models, dynamical systems, environment, geophysics, hurricanes, Kalman filter, Kerry Emanuel, Lévy flights, Lorenz, Markov chain random fields, mathematics, mathematics education, maths, MCMC, mesh models, meteorological models, meteorology, model-free forecasting, Monte Carlo Statistical Methods, numerical analysis, numerical software, oceanography, open data, open source scientific software, physics, random walk processes, random walks, science, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, time series | 1 Comment

“Holy crap – an actual book!”

Posted on 10 August 2016 by ecoquant

Originally posted on mathbabe:
Yo, everyone! The final version of my book now exists, and I have exactly one copy! Here’s my editor, Amanda Cook, holding it yesterday when we met for beers: Here’s my son holding it: He’s offered…

Posted in American Association for the Advancement of Science, Buckminster Fuller, business, citizen science, citizenship, civilization, complex systems, confirmation bias, data science, data streams, deep recurrent neural networks, denial, economics, education, engineering, ethics, evidence, Internet, investing, life purpose, machine learning, mathematical publishing, mathematics, mathematics education, maths, moral leadership, multivariate statistics, numerical software, numerics, obfuscating data, organizational failures, politics, population biology, prediction, prediction markets, privacy, quantitative biology, quantitative ecology, rationality, reason, reasonableness, rhetoric, risk, Schnabel census, smart data, sociology, statistical dependence, statistics, the right to be and act stupid, the right to know, the value of financial assets, transparency, UU Humanists | Leave a comment

A model of an electrical grid: A vision

Posted on 3 August 2016 by ecoquant

Many people seem to view the electrical grid of the future being much like the present one. I think a lot about networks, because of my job. And I especially think a lot about network topologies, although primarily concerning the … Continue reading →

Posted in abstraction, American Meteorological Association, anomaly detection, Anthropocene, Bloomberg New Energy Finance, BNEF, Boston, bridge to somewhere, Buckminster Fuller, Canettes Blues Band, clean disruption, climate business, climate economics, complex systems, corporate supply chains, decentralized electric power generation, decentralized energy, demand-side solutions, differential equations, distributed generation, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy storage, energy utilities, engineering, extended supply chains, green tech, grid defection, Hermann Scheer, Hyper Anthropocene, investment in wind and solar energy, ISO-NE, Kalman filter, kriging, Lawrence Berkeley National Laboratory, leaving fossil fuels in the ground, Lenny Smith, local generation, marginal energy sources, Massachusetts Clean Energy Center, Mathematics and Climate Research Network, mesh models, meteorology, microgrids, networks, New England, New York State, open data, organizational failures, pipelines, planning, prediction markets, public utility commissions, PUCs, rate of return regulation, rationality, reason, reasonableness, regime shifts, regulatory capture, resiliency, risk, Sankey diagram, smart data, solar domination, solar energy, solar power, Spaceship Earth, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, stranded assets, supply chains, sustainability, the energy of the people, the green century, the value of financial assets, thermodynamics, time series, Tony Seba, utility company death spiral, wave equations, wind energy, wind power, zero carbon | Leave a comment

“Stochastic Parameterization: Towards a new view of weather and climate models”

Posted on 24 July 2016 by ecoquant

Judith Berner, Ulrich Achatz, Lauriane Batté, Lisa Bengtsson, Alvaro De La Cámara, Hannah M. Christensen, Matteo Colangeli, Danielle R. B. Coleman, Daan Crommelin, Stamen I. Dolaptchiev, Christian L.E. Franzke, Petra Friederichs, Peter Imkeller, Heikki Järvinen, Stephan Juricke, Vassili Kitsios, François … Continue reading →

Posted in biology, climate models, complex systems, convergent cross-mapping, data science, dynamical systems, ecology, Ethan Deyle, Floris Takens, George Sughihara, Hao Ye, likelihood-free, Lorenz, mathematics, meteorological models, model-free forecasting, physics, population biology, population dynamics, quantitative biology, quantitative ecology, Scripps Institution of Oceanography, state-space models, statistical dependence, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, time series, Victor Brovkin | 4 Comments

On Smart Data

Posted on 11 June 2016 by ecoquant

One of the things I find surprising, if not astonishing, is that in the rush to embrace Big Data, a lot of learning and statistical technique has been left apparently discarded along the way. I’m hardly the first to point … Continue reading →

Posted in Akaike Information Criterion, Bayes, Bayesian, Bayesian inversion, big data, bigmemory package for R, changepoint detection, data science, data streams, dlm package, dynamic generalized linear models, dynamic linear models, dynamical systems, Generalize Additive Models, generalized linear models, information theoretic statistics, Kalman filter, linear algebra, logistic regression, machine learning, Markov Chain Monte Carlo, mathematics, mathematics education, maths, maximum likelihood, MCMC, Monte Carlo Statistical Methods, multivariate statistics, numerical analysis, numerical software, numerics, quantitative biology, quantitative ecology, rationality, reasonableness, sampling, smart data, state-space models, statistical dependence, statistics, the right to know, time series | Leave a comment

Cory Lesmeister’s treatment of Simson’s Paradox (at “Fear and Loathing in Data Science”)

Posted on 7 May 2016 by ecoquant

(Updated 2016-05-08, to provide reference for plateaus of ML functions in vicinity of MLE.) Simpson’s Paradox is one of those phenomena of data which really give Statistics a substance and a role, beyond the roles it inherits from, say, theoretical … Continue reading →

Posted in Akaike Information Criterion, approximate Bayesian computation, Bayes, Bayesian, evidence, Frequentist, games of chance, information theoretic statistics, Kalman filter, likelihood-free, mathematics, maths, maximum likelihood, Monte Carlo Statistical Methods, probabilistic programming, rationality, Rauch-Tung-Striebel, Simpson's Paradox, state-space models, statistical dependence, statistics, stochastics | Leave a comment

“Lucky d20” (by Tamino, with my reblogging comments)

Posted on 1 May 2016 by ecoquant

Originally posted on Open Mind:
What with talk of killer heat waves, droughts, floods, etc. etc., this blog tends to get pretty serious. When it does, we don’t deal with happy prospects, but with the danger of worldwide catastrophe. But…

Posted in Bayes, Bayesian, card decks, card draws, card games, chance, D&D, Dungeons and Dragons, games of chance, mathematics, maths, Monte Carlo Statistical Methods, probability, statistical dependence, statistics, stochastic algorithms, stochastics, Wizards of the Coast | Leave a comment

Mr Buffett bets the farm

Posted on 28 April 2016 by ecoquant

From Dr James Hansen’s blog, of today. So, Mr. Buffett, I am heartened by the words in your last annual report, where you conclude that continued inaction on climate change “is foolhardy.” You wrote: “Call this Noah’s Law: If an … Continue reading →

Posted in adaptation, Anthropocene, Bloomberg New Energy Finance, BNEF, climate change, climate disruption, climate justice, corporate litigation on damage from fossil fuel emissions, environment, extended supply chains, games of chance, geophysics, global warming, greenhouse gases, hurricanes, Hyper Anthropocene, insurance, investing, James Hansen, liberal climate deniers, meteorology, oceanography, physics, rate of return regulation, rationality, reasonableness, science, statistical dependence, statistics, sustainability, the right to be and act stupid, the tragedy of our present civilization, the value of financial assets, Wally Broecker, Warren Buffett, zero carbon | Leave a comment

p-values and hypothesis tests: the Bayesian(s) rule

Posted on 7 March 2016 by ecoquant

The American Statistical Association of which I am a longtime member issued an important statement today which will hopefully move statistical practice in engineering and especially in the sciences away from the misleading practice of using p-values and hypothesis tests. … Continue reading →

Posted in approximate Bayesian computation, arXiv, Bayes, Bayesian, Bayesian inversion, bollocks, Christian Robert, climate, complex systems, data science, Frequentist, information theoretic statistics, likelihood-free, Markov Chain Monte Carlo, MCMC, Monte Carlo Statistical Methods, population biology, rationality, reasonableness, science, scientific publishing, statistical dependence, statistics, stochastics, Student t distribution | Leave a comment

Ah, Hypergeometric!

Posted on 8 February 2016 by ecoquant

(“Ah, Hypergeometric!” To be said with the same resignation and acceptance as in “I’ll burn my books–Ah, Mephistopheles!” from Faust.)😉 Dr John Cook, eminent all ’round statistician (with a specialty in biostatistics) and statistical consultant, took up a comment I … Continue reading →

Posted in card decks, card draws, card games, games of chance, John Cook, mathematics, maths, probability, sampling, sampling without replacement, statistical dependence | Leave a comment