Category Archives: time series

`Letter to Lamar Smith’

On Ed Hawkins’ blog. The Committee on Science, Space & Technology of the US House of Representatives conducts regular evidence hearings on various science topics. On Wednesday 29th March, there is a hearing on “Climate science: assumptions, policy implications, and … Continue reading

Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, anemic data, anomaly detection, Anthropocene, Ben Santer, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, changepoint detection, climate, climate change, climate data, climate disruption, Climate Lab Book, climate zombies, dependent data, environment, fossil fuel divestment, geophysics, global warming, greenhouse gases, Humans have a lot to answer for, Hyper Anthropocene, leaving fossil fuels in the ground, meteorology, MIchael Mann, Our Children's Trust, physics, science, smoothing, statistical dependence, the right to be and act stupid, the right to know, the tragedy of our present civilization, the value of financial assets, time series | Leave a comment

Energy Consumption with Air Source Heat Pumps and Water Heater

Once nice thing about having a net metered solar PV array is that, with a little diligence, you can figure out how much electricity your household is consuming each day, or at finer resolution if you like (*). Below is … Continue reading

Posted in Anthropocene, Bloomberg New Energy Finance, clean disruption, CleanTechnica, climate economics, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, efficiency, energy reduction, engineering, global warming, Hyper Anthropocene, ISO-NE, local self reliance, New England, smart data, solar democracy, solar domination, solar energy, statistics, time series, Tony Seba, zero carbon | Leave a comment

“All models are wrong. Some models are useful.” — George Box

(Image courtesy of the Damien Garcia.) As a statistician and quant, I’ve thought hard about that oft-cited Boxism. I’m not sure I agree. It’s not that there is such a thing as a perfect model, or correct model, whatever in … Continue reading

Posted in abstraction, American Association for the Advancement of Science, astronomy, astrophysics, mathematics, model-free forecasting, numerics, perceptions, physical materialism, physics, rationality, reason, reasonableness, science, spatial statistics, splines, statistics, the right to know, theoretical physics, time series | Leave a comment

Just a lil’ bit o’ a drought … Nothing to be alarmed about … (!)

Posted in adaptation, American Meteorological Association, AMETSOC, Anthropocene, atmosphere, climate change, climate data, climate disruption, drought, environment, fluid dynamics, global warming, greenhouse gases, hydrology, Hyper Anthropocene, leaving fossil fuels in the ground, meteorology, quantitative ecology, Spaceship Earth, statistics, time series, water, water vapor, WHOI, zero carbon | Leave a comment

“Predicting annual temperatures a year ahead” (Dr Gavin Schmidt at REALCLIMATE)

Dr Schmidt is essentially betting that the trend, seen as a random variable, will regress towards the smooth mean. I have a post at Nate Silver’s 538 site on how we can predict annual surface temperature anomalies based on El … Continue reading

Posted in American Meteorological Association, American Statistical Association, AMETSOC, anomaly detection, Anthropocene, changepoint detection, climate change, climate data, climate disruption, climate education, climate models, ecology, environment, forecasting, geophysics, global warming, Hyper Anthropocene, meteorology, oceanography, physics, regression toward the mean, science, statistics, time series | Leave a comment

Bastardi’s Bust

Famous climate denialist Joe Bastari of WeatherBELL Analytics LLC, formerly of Accuweather.com made a prediction on Arctic ice recovery back in 2010 (when at AccuWeather), and observations have since made his “studies” laughable. I have heard his colleague, Joseph D’Aleo … Continue reading

Posted in Accuweather, American Meteorological Association, anomaly detection, Anthropocene, Arctic, climate change, climate disruption, climate models, coasts, ecology, environment, evidence, global warming, Hyper Anthropocene, ice sheet dynamics, meteorology, NOAA, science denier, shorelines, statistics, Stefan Rahmstorf, the right to be and act stupid, the right to know, the stack of lies, the tragedy of our present civilization, time series | Leave a comment

“Naïve empiricism and what theory suggests about errors in observed global warming”

A post from one of my favorite statistics-oriented bloggers, Variable Variability, dealing with a subject too casually passed over. See Naïve empiricism and what theory suggests about errors in observed global warming.

Posted in Anthropocene, climate, climate change, climate data, climate disruption, confirmation bias, geophysics, rationality, statistics, time series, Variable Variability, Victor Venema | Leave a comment

Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories

I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random walk in a parameter space. … Continue reading

Posted in American Meteorological Association, American Statistical Association, AMETSOC, Arthur Charpentier, atmosphere, diffusion, diffusion processes, dynamic linear models, dynamical systems, environment, geophysics, hurricanes, Kalman filter, Kerry Emanuel, Lévy flights, Lorenz, Markov chain random fields, mathematics, mathematics education, maths, MCMC, mesh models, meteorological models, meteorology, model-free forecasting, Monte Carlo Statistical Methods, numerical analysis, numerical software, oceanography, open data, open source scientific software, physics, random walk processes, random walks, science, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, time series | Leave a comment

A model of an electrical grid: A vision

Many people seem to view the electrical grid of the future being much like the present one. I think a lot about networks, because of my job. And I especially think a lot about network topologies, although primarily concerning the … Continue reading

Posted in abstraction, American Meteorological Association, anomaly detection, Anthropocene, Bloomberg New Energy Finance, BNEF, Boston, bridge to somewhere, Buckminster Fuller, Canettes Blues Band, clean disruption, climate business, climate economics, complex systems, corporate supply chains, decentralized electric power generation, decentralized energy, demand-side solutions, differential equations, distributed generation, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy storage, energy utilities, engineering, extended supply chains, green tech, grid defection, Hermann Scheer, Hyper Anthropocene, investment in wind and solar energy, ISO-NE, Kalman filter, kriging, Lawrence Berkeley National Laboratory, leaving fossil fuels in the ground, Lenny Smith, local generation, marginal energy sources, Massachusetts Clean Energy Center, Mathematics and Climate Research Network, mesh models, meteorology, microgrids, networks, New England, New York State, open data, organizational failures, pipelines, planning, prediction markets, public utility commissions, PUCs, rate of return regulation, rationality, reason, reasonableness, regime shifts, regulatory capture, resiliency, risk, Sankey diagram, smart data, solar domination, solar energy, solar power, Spaceship Earth, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, stranded assets, supply chains, sustainability, the energy of the people, the green century, the value of financial assets, thermodynamics, time series, Tony Seba, utility company death spiral, wave equations, wind energy, wind power, zero carbon | Leave a comment

Bayesian blocks via PELT in R

The Bayesian blocks algorithm of Scargle, Jackson, Norris, and Chiang has an enthusiastic user community in astrostatistics, in data mining, and among some in machine learning. It is a dynamic programming algorithm (see VanderPlas referenced below) and, so, exhibits optimality … Continue reading

Posted in American Statistical Association, AMETSOC, anomaly detection, astrophysics, Cauchy distribution, changepoint detection, engineering, geophysics, multivariate statistics, numerical analysis, numerical software, numerics, oceanography, population biology, population dynamics, Python 3, quantitative biology, quantitative ecology, R, Scargle, spatial statistics, square wave approximation, statistics, stepwise approximation, time series, Woods Hole Oceanographic Institution | Leave a comment

“Full-depth Ocean Heat Content” reblog

This is a re-blog of an excellent post at And Then There’s Physics, titled Full-depth OHC or, expanded, “full-depth ocean heat content”. Since my holiday is now over, I thought I might briefly comment on a recent paper by Cheng … Continue reading

Posted in Anthropocene, climate, climate change, climate data, climate disruption, climate models, computation, differential equations, ensembles, environment, fluid dynamics, forecasting, geophysics, global warming, greenhouse gases, Hyper Anthropocene, Lorenz, Mathematics and Climate Research Network, model comparison, NOAA, oceanography, physics, science, statistics, theoretical physics, thermodynamics, time series | Leave a comment

“Stochastic Parameterization: Towards a new view of weather and climate models”

Judith Berner, Ulrich Achatz, Lauriane Batté, Lisa Bengtsson, Alvaro De La Cámara, Hannah M. Christensen, Matteo Colangeli, Danielle R. B. Coleman, Daan Crommelin, Stamen I. Dolaptchiev, Christian L.E. Franzke, Petra Friederichs, Peter Imkeller, Heikki Järvinen, Stephan Juricke, Vassili Kitsios, François … Continue reading

Posted in biology, climate models, complex systems, convergent cross-mapping, data science, dynamical systems, ecology, Ethan Deyle, Floris Takens, George Sughihara, Hao Ye, likelihood-free, Lorenz, mathematics, meteorological models, model-free forecasting, physics, population biology, population dynamics, quantitative biology, quantitative ecology, Scripps Institution of Oceanography, state-space models, statistical dependence, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, time series, Victor Brovkin | 4 Comments

On Smart Data

One of the things I find surprising, if not astonishing, is that in the rush to embrace Big Data, a lot of learning and statistical technique has been left apparently discarded along the way. I’m hardly the first to point … Continue reading

Posted in Akaike Information Criterion, Bayes, Bayesian, Bayesian inversion, big data, bigmemory package for R, changepoint detection, data science, data streams, dlm package, dynamic generalized linear models, dynamic linear models, dynamical systems, Generalize Additive Models, generalized linear models, information theoretic statistics, Kalman filter, linear algebra, logistic regression, machine learning, Markov Chain Monte Carlo, mathematics, mathematics education, maths, maximum likelihood, MCMC, Monte Carlo Statistical Methods, multivariate statistics, numerical analysis, numerical software, numerics, quantitative biology, quantitative ecology, rationality, reasonableness, sampling, smart data, state-space models, statistical dependence, statistics, the right to know, time series | Leave a comment

“Catching long tail distribution” (Ted Dunning)

One of the best presentations on what can happen if someone takes a naive approach to network data. It also highlights what is, to my mind, the greatly underappreciated t-distribution, which is typically only used in connection with frequentist Student … Continue reading

Posted in Cauchy distribution, complex systems, data science, Lévy flights, leptokurtic, mathematics, maths, networks, physics, population biology, population dynamics, regime shifts, sampling, statistics, Student t distribution, time series | Leave a comment

Six cases of models

The previous post included an attempt to explain land surface temperatures as estimated by the BEST project using a dynamic linear model including regressions on both quarterly CO2 concentrations and ocean heat content. The idea was to check the explanatory … Continue reading

Posted in AMETSOC, anemic data, Anthropocene, astrophysics, Bayesian, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, climate, climate change, climate data, climate disruption, climate models, dlm package, dynamic linear models, dynamical systems, environment, fossil fuels, geophysics, Giovanni Petris, global warming, greenhouse gases, Hyper Anthropocene, information theoretic statistics, maths, maximum likelihood, meteorology, model comparison, numerical software, Patrizia Campagnoli, Rauch-Tung-Striebel, Sonia Petrone, state-space models, stochastic algorithms, stochastic search, SVD, time series | 1 Comment

On Munshi mush

(Slightly updated on 2016-06-11.) Professor Emeritus Jamal Munshi of Sonoma State University has papers recently cited in science denier circles as evidence that the conventional associations between mean global surface temperature and cumulative carbon emissions are, well, bunk, due to … Continue reading

Posted in Bayes, Bayesian, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, cat1, climate, climate change, climate data, climate education, climate models, convergent cross-mapping, dynamic linear models, ecology, ENSO, environment, Ethan Deyle, evidence, geophysics, George Sughihara, global warming, greenhouse gases, information theoretic statistics, Kalman filter, mathematics, maths, meteorology, model comparison, NOAA, oceanography, prediction, state-space models, statistics, Takens embedding theorem, Techno Utopias, the right to know, theoretical physics, time series, zero carbon | 1 Comment

10+ kilowatts (!) from a PV with 29 SunPower panels designed and installed by RevoluSun

(Click on image to see larger figure. Use your browser Back Button to return to blog.)

Posted in Anthropocene, bridge to somewhere, clean disruption, decentralized electric power generation, decentralized energy, destructive economic development, distributed generation, electricity, electricity markets, fossil fuel divestment, grid defection, Hyper Anthropocene, investment in wind and solar energy, local generation, microgrids, rate of return regulation, rationality, reasonableness, RevoluSun, Sankey diagram, solar energy, solar power, SolarPV.tv, Spaceship Earth, SunPower, sustainability, the energy of the people, the green century, the value of financial assets, time series, Tony Seba, zero carbon | 1 Comment

Clear of all trees

One drawback of solar panels at our home site is a significant stand of conifers to our southwest. (Click image for a larger picture. Use browser Back Button to return to blog.) It’s clear when the trees are casting shadows … Continue reading

Posted in adaptation, Anthropocene, Bloomberg New Energy Finance, BNEF, bridge to somewhere, clean disruption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, distributed generation, electricity, energy, energy reduction, fossil fuel divestment, Hyper Anthropocene, investment in wind and solar energy, rationality, reasonableness, Sankey diagram, scattering, solar domination, solar energy, solar power, SolarPV.tv, the energy of the people, the green century, time series, utility company death spiral, zero carbon | Leave a comment

Gavin Simpson updates his temperature analysis

See the very interesting discussion at his blog, From the bottom of the heap. It would be nice to see some information theoretic measures on these results, though.

Posted in AMETSOC, Anthropocene, astrophysics, Berkeley Earth Surface Temperature project, carbon dioxide, changepoint detection, climate, climate change, climate data, climate disruption, climate models, ecology, environment, evidence, Gavin Simpson, Generalize Additive Models, geophysics, global warming, HadCRUT4, hiatus, Hyper Anthropocene, information theoretic statistics, Kalman filter, maths, meteorology, numerical analysis, R, rationality, reasonableness, splines, time series | Leave a comment

HadCRUT4 and GISTEMP series filtered and estimated with simple RTS model

Happy Vernal Equinox! This post has been updated today with some of the equations which correspond to the models. An assessment of whether or not there was a meaningful slowdown or “hiatus” in global warming, was recently discussed by Tamino … Continue reading

Posted in AMETSOC, anemic data, Bayesian, boosting, bridge to somewhere, cat1, changepoint detection, climate, climate change, climate data, climate disruption, climate models, complex systems, computation, data science, dynamical systems, geophysics, George Sughihara, global warming, hiatus, information theoretic statistics, machine learning, maths, meteorology, MIchael Mann, multivariate statistics, physics, prediction, Principles of Planetary Climate, rationality, reasonableness, regime shifts, sea level rise, time series | 2 Comments

On friction and the duplicity

(Hat tip to Peter Sinclair at Climate Denial Crock of the Week.) Has Senator Cruz called Dr Carl Mears (video) of Remote Sensing Systems, the maker and interpreter of the sensor Senator Cruz used for his Spencer-Christy-Curry carnival? No. Of … Continue reading

Posted in AMETSOC, anemic data, Berkeley Earth Surface Temperature project, BEST, climate, climate change, climate data, climate disruption, confirmation bias, corruption, denial, disingenuity, ecology, evidence, fear uncertainty and doubt, geophysics, global warming, greenhouse gases, hiatus, Hyper Anthropocene, ignorance, meteorology, model comparison, NCAR, NOAA, obfuscating data, oceanography, physics, rationality, reasonableness, statistics, time series | Leave a comment

“Causal feedbacks in climate change”

Today I was reviewing and re-reading the nonlinear time series technical literature I have, seeking ideas on how to go about using the statistical ensemble learning technique called “boosting” with them. (See the very nice book, R. E. Schapire, Y. … Continue reading

Posted in Anthropocene, boosting, Carbon Cycle, carbon dioxide, Carbon Worshipers, cat1, climate, climate change, climate data, climate disruption, complex systems, convergent cross-mapping, denial, differential equations, diffusion processes, dynamical systems, ecology, Egbert van Nes, empirical likelihood, ensembles, environment, Ethan Deyle, Floris Takens, forecasting, fossil fuels, geophysics, George Sughihara, global warming, greenhouse gases, Hao Ye, machine learning, Maren Scheffer, mathematics, maths, meteorology, physics, rationality, reasonableness, science, state-space models, Takens embedding theorem, time series, Timothy Lenton, Victor Brovkin | 1 Comment

Hottest Year on Record

Originally posted on Open Mind:
Back when Richard Muller announced the formation of the Berkeley Earth Surface Temperature project, those who deny the danger from global warming were thrilled. They thought the Berkeley project would prove once and for all…

Posted in AMETSOC, Anthropocene, Berkeley, Berkeley Earth Surface Temperature project, BEST, BLUE, carbon dioxide, climate, climate change, climate data, climate disruption, climate education, climate zombies, environment, evidence, geophysics, global warming, Hyper Anthropocene, James Hansen, kriging, meteorology, NCAR, NOAA, physics, Principles of Planetary Climate, rationality, reasonableness, Richard Muller, Robert Rohde, science, science education, Tamino, the right to know, time series, University of California Berkeley | Leave a comment

After the Decade of Dithering, the Deadly Twenties

In a recent post, after reviewing the extreme Arctic warming event of late 2015, Professor John Baez quotes an earlier interview with Dr Gregory Benford, who is arguing for a geoengineering effort to restore the frozen Arctic. I do not … Continue reading

Posted in adaptation, AMOC, Arctic, chance, changepoint detection, climate, climate change, climate disruption, critical slowing down, ecology, engineering, geoengineering, global warming, greenhouse gases, Hyper Anthropocene, ignorance, James Hansen, MIchael Mann, mitigation, oceanography, physics, politics, rationality, reasonableness, regime shifts, science, science education, state-space models, statistics, the right to know, thermohaline circulation, time series | Leave a comment

Not too shabby: “What’s warming the world” (Bloomberg Business), and “The siege of Miami” (The New Yorker)

What’s warming the world Infographic allowing the visitor to overlay time series of candidate causes for global warming, and thereby permitting them to draw their own conclusions. And Elizabeth Kolbert’s piece in The New Yorker, brings home the contradictions and … Continue reading

Posted in Anthropocene, business, climate change, climate data, climate zombies, complex systems, critical slowing down, denial, disingenuity, economics, environment, evidence, fossil fuels, global warming, greenhouse gases, Hyper Anthropocene, mitigation, model comparison, time series | Leave a comment

All I do is complain, complain …

I was reviewing a presentation given as part of a short course in the machine learning genre today, and happened across the following two bullets, under the heading “Strictly Stationary Processes”: Predicting a time series is possible if and only … Continue reading

Posted in bifurcations, chaos, citizen science, convergent cross-mapping, dynamic linear models, dynamical systems, engineering, Floris Takens, generalized linear models, geophysics, George Sughihara, ignorance, Lenny Smith, Lorenz, mathematics, maths, meteorology, prediction, probability, rationality, reasonableness, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, the right to know, time series | 1 Comment

dynamic linear model applied to sea-level-rise anomalies

I spent much of the data working up a function for level+trend dynamic linear modeling based upon the dlm package by Petris, Petrone, and Campagnoli, while trying some calculations and code for regime shift detection. One of the test cases … Continue reading

Posted in Bayesian, citizen science, climate change, climate data, climate disruption, dynamic linear models, floods, forecasting, Frequentist, global warming, icesheets, information theoretic statistics, Kalman filter, meteorology, open data, sea level rise, state-space models, statistics, time series | Leave a comment

El Nino In A Can – Dan’s Wild Wild Science Journal – AGU Blogosphere

Click the image above to see a video from the GFDL CM2.6 climate model. This is NOT this year’s El Nino. When you start a climate model in which the ocean and the land and atmosphere can inte… Source: El … Continue reading

Posted in AMETSOC, astrophysics, climate, climate change, climate models, computation, Dan Satterfield, differential equations, diffusion, diffusion processes, dynamical systems, ENSO, environment, forecasting, geophysics, global warming, Hyper Anthropocene, Kerry Emanuel, mathematics, maths, mesh models, meteorology, model comparison, NASA, NCAR, NOAA, numerical analysis, oceanography, physics, Principles of Planetary Climate, rationality, Ray Pierrehumbert, reasonableness, science, Spaceship Earth, stochastics, supercomputers, the right to know, thermodynamics, time series | Leave a comment

Thoughts on “Regime Shift?”

John Baez at The Azimuth Project opened a discussion on the recent paper by Reid, et al Philip C. Reid et al, Global impacts of the 1980s regime shift on the Earth’s climate and systems, Global Change Biology, 2015. I … Continue reading

Posted in Bayesian, changepoint detection, climate change, climate disruption, climate models, dynamic linear models, ecology, ensembles, environment, global warming, population biology, Rauch-Tung-Striebel, regime shifts, state-space models, stochastic algorithms, time series | Leave a comment

Southern Oscillation (SOI) correlated with Outgoing Longwave Radiation (OLR)

To the climate community this is nothing at all new, but I spotted these time series today and thought they would make a nice exhibit on how something people have direct control over, greenhouse gas emissions, affect a “teleconnection mechanism” … Continue reading

Posted in AMETSOC, bifurcations, carbon dioxide, climate, climate change, climate disruption, climate models, Dan Satterfield, differential equations, dynamic linear models, dynamical systems, ENSO, environment, forecasting, generalized linear models, geophysics, global warming, greenhouse gases, IPCC, Mathematica, mathematics, maths, meteorology, NCAR, NOAA, numerical software, oceanography, open data, physics, population biology, Principles of Planetary Climate, rationality, reasonableness, science, Spaceship Earth, state-space models, thermodynamics, time series | Leave a comment