Category Archives: dynamic linear models

Baseload is an intellectual crutch for engineers and utility managers who cannot think dynamically

This is an awesome presentation by Professor Joshua Pearce of Michigan Technological University. (h/t Peter Sinclair’s Climate Denial Crock of the Week) The same idea, that “baseload is a shortcut for engineers who can’t think dynamically”, was similar in the … Continue reading

Posted in American Solar Energy Society, an ignorant American public, Bloomberg Green, Bloomberg New Energy Finance, bridge to somewhere, CleanTechnica, control theory, controls theory, decentralized electric power generation, decentralized energy, differential equations, dynamic linear models, dynamical systems, electrical energy engineering, electrical energy storage, electricity, Kalman filter, optimization, photovoltaics, rate of return regulation, solar domination, solar energy, solar revolution, stochastic algorithms, utility company death spiral, wind energy, wind power, zero carbon | Tagged , , , , , , , | Leave a comment

Phase Plane plots of COVID-19 deaths with uncertainties

I. Introduction. It’s time to fulfill the promise made in “Phase plane plots of COVID-19 deaths“, a blog post from 2nd May 2020, and produce the same with uncertainty clouds about the functional trajectories(*). To begin, here are some assumptions … Continue reading

Posted in American Statistical Association, Andrew Harvey, anomaly detection, count data regression, COVID-19, dependent data, dlm package, Durbin and Koopman, dynamic linear models, epidemiology, filtering, forecasting, Kalman filter, LaTeX, model-free forecasting, Monte Carlo Statistical Methods, numerical algorithms, numerical linear algebra, population biology, population dynamics, prediction, R, R statistical programming language, regression, statistical learning, stochastic algorithms | Tagged | Leave a comment

Stream flow and P-splines: Using built-in estimates for smoothing

Mother Brook in Dedham Massachusetts was the first man-made canal in the United States. Dug in 1639, it connects the Charles River at Dedham, to the Neponset River in the Hyde Park section of Boston. It was originally an important … Continue reading

Posted in American Statistical Association, citizen data, citizen science, Clausius-Clapeyron equation, Commonwealth of Massachusetts, cross-validation, data science, dependent data, descriptive statistics, dynamic linear models, empirical likelihood, environment, flooding, floods, Grant Foster, hydrology, likelihood-free, meteorological models, model-free forecasting, non-mechanistic modeling, non-parametric, non-parametric model, non-parametric statistics, numerical algorithms, precipitation, quantitative ecology, statistical dependence, statistical series, stream flow, Tamino, the bootstrap, time series, water vapor | 2 Comments

climate model democracy

“One of the most interesting things about the MIP ensembles is that the mean of all the models generally has higher skill than any individual model.” We hold these truths to be self-evident, that all models are created equal, that … Continue reading

Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, Anthropocene, attribution, Bayesian model averaging, Bloomberg, citizen science, climate, climate business, climate change, climate data, climate disruption, climate education, climate justice, Climate Lab Book, climate models, coastal communities, coastal investment risks, complex systems, differential equations, disruption, dynamic linear models, dynamical systems, ecology, emergent organization, ensemble methods, ensemble models, ensembles, Eric Rignot, evidence, fear uncertainty and doubt, FEMA, forecasting, free flow of labor, global warming, greenhouse gases, greenwashing, Humans have a lot to answer for, Hyper Anthropocene, Jennifer Francis, Joe Romm, Kevin Anderson, Lévy flights, LBNL, leaving fossil fuels in the ground, liberal climate deniers, mathematics, mathematics education, model-free forecasting, multivariate adaptive regression splines, National Center for Atmospheric Research, obfuscating data, oceanography, open source scientific software, optimization, perceptrons, philosophy of science, phytoplankton | Leave a comment

When linear systems can’t be solved by linear means

Linear systems of equations and their solution form the cornerstone of much Engineering and Science. Linear algebra is a paragon of Mathematics in the sense that its theory is what mathematicians try to emulate when they develop theory for many … Continue reading

Posted in Calculus, dynamic linear models, mathematics, maths, nloptr, numerical algorithms, numerical analysis, numerical linear algebra, numerics, SVD | Leave a comment

on nonlinear dynamics of hordes of people

I spent a bit of last week at a symposium honoring the work of Charney and Lorenz in fluid dynamics. I am no serious student of fluid dynamics. I have a friend, Klaus, an engineer, who is, and makes a … Continue reading

Posted in Anthropocene, bifurcations, biology, Carl Safina, causation, complex systems, dynamic generalized linear models, dynamic linear models, dynamical systems, ecological services, ecology, Emily Shuckburgh, finance, Floris Takens, fluid dynamics, fluid eddies, games of chance, Hyper Anthropocene, investments, Lenny Smith, Lorenz, nonlinear, numerical algorithms, numerical analysis, politics, population biology, population dynamics, prediction markets, Principles of Planetary Climate, public transport, Ray Pierrehumbert, risk, sampling networks, sustainability, Timothy Lenton, Yale University Statistics Department, zero carbon, ``The tide is risin'/And so are we'' | 1 Comment

Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories

(Slight update, 28th June 2020.) I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random … Continue reading

Posted in American Meteorological Association, American Statistical Association, AMETSOC, Arthur Charpentier, atmosphere, diffusion, diffusion processes, dynamic linear models, dynamical systems, environment, geophysics, hurricanes, Kalman filter, Kerry Emanuel, Lévy flights, Lorenz, Markov chain random fields, mathematics, mathematics education, maths, MCMC, mesh models, meteorological models, meteorology, model-free forecasting, Monte Carlo Statistical Methods, numerical analysis, numerical software, oceanography, open data, open source scientific software, physics, random walk processes, random walks, science, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, time series | 1 Comment

On Smart Data

One of the things I find surprising, if not astonishing, is that in the rush to embrace Big Data, a lot of learning and statistical technique has been left apparently discarded along the way. I’m hardly the first to point … Continue reading

Posted in Akaike Information Criterion, Bayes, Bayesian, Bayesian inversion, big data, bigmemory package for R, changepoint detection, data science, data streams, dlm package, dynamic generalized linear models, dynamic linear models, dynamical systems, Generalize Additive Models, generalized linear models, information theoretic statistics, Kalman filter, linear algebra, logistic regression, machine learning, Markov Chain Monte Carlo, mathematics, mathematics education, maths, maximum likelihood, MCMC, Monte Carlo Statistical Methods, multivariate statistics, numerical analysis, numerical software, numerics, quantitative biology, quantitative ecology, rationality, reasonableness, sampling, smart data, state-space models, statistical dependence, statistics, the right to know, time series | Leave a comment

Six cases of models

The previous post included an attempt to explain land surface temperatures as estimated by the BEST project using a dynamic linear model including regressions on both quarterly CO2 concentrations and ocean heat content. The idea was to check the explanatory … Continue reading

Posted in AMETSOC, anemic data, Anthropocene, astrophysics, Bayesian, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, climate, climate change, climate data, climate disruption, climate models, dlm package, dynamic linear models, dynamical systems, environment, fossil fuels, geophysics, Giovanni Petris, global warming, greenhouse gases, Hyper Anthropocene, information theoretic statistics, maths, maximum likelihood, meteorology, model comparison, numerical software, Patrizia Campagnoli, Rauch-Tung-Striebel, Sonia Petrone, state-space models, stochastic algorithms, stochastic search, SVD, time series | 1 Comment

On Munshi mush

(Slightly updated on 2016-06-11.) Professor Emeritus Jamal Munshi of Sonoma State University has papers recently cited in science denier circles as evidence that the conventional associations between mean global surface temperature and cumulative carbon emissions are, well, bunk, due to … Continue reading

Posted in Bayes, Bayesian, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, cat1, climate, climate change, climate data, climate education, climate models, convergent cross-mapping, dynamic linear models, ecology, ENSO, environment, Ethan Deyle, evidence, geophysics, George Sughihara, global warming, greenhouse gases, information theoretic statistics, Kalman filter, mathematics, maths, meteorology, model comparison, NOAA, oceanography, prediction, state-space models, statistics, Takens embedding theorem, Techno Utopias, the right to know, theoretical physics, time series, zero carbon | 1 Comment

“The Myth of the 1970s Global Cooling Consensus”

“The Myth of the 1970s Global Cooling Scientific Consensus“, T. C. Peterson, W. M. Connolley, J. Fleck, http://dx.doi.org/10.1175/2008BAMS2370.1. Abstract Climate science as we know it today did not exist in the 1960s and 1970s. The integrated enterprise embodied in the … Continue reading

Posted in AMETSOC, Anthropocene, carbon dioxide, citizen science, climate, climate change, climate data, climate disruption, climate education, climate zombies, coastal communities, differential equations, dynamic linear models, dynamical systems, ecology, environment, fluid dynamics, fossil fuel divestment, fossil fuels, geophysics, global warming, greenhouse gases, Hyper Anthropocene, ice sheet dynamics, investing | Leave a comment

Phytoplankton-delineated oceanic eddies near Antarctica

Excerpt, from NASA: Phytoplankton are the grass of the sea. They are floating, drifting, plant-like organisms that harness the energy of the Sun, mix it with carbon dioxide that they take from the atmosphere, and turn it into carbohydrates and … Continue reading

Posted in AMETSOC, Antarctica, Arctic, bacteria, Carbon Cycle, complex systems, differential equations, diffusion, diffusion processes, dynamic linear models, dynamical systems, Emily Shuckburgh, environment, fluid dynamics, geophysics, GLMs, John Marshall, marine biology, Mathematics and Climate Research Network, NASA, numerical analysis, numerical software, oceanic eddies, oceanography, physics, phytoplankton, science, thermohaline circulation, WHOI, Woods Hole Oceanographic Institution | Leave a comment

All I do is complain, complain …

I was reviewing a presentation given as part of a short course in the machine learning genre today, and happened across the following two bullets, under the heading “Strictly Stationary Processes”: Predicting a time series is possible if and only … Continue reading

Posted in bifurcations, chaos, citizen science, convergent cross-mapping, dynamic linear models, dynamical systems, engineering, Floris Takens, generalized linear models, geophysics, George Sughihara, ignorance, Lenny Smith, Lorenz, mathematics, maths, meteorology, prediction, probability, rationality, reasonableness, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, the right to know, time series | 1 Comment

dynamic linear model applied to sea-level-rise anomalies

I spent much of the data working up a function for level+trend dynamic linear modeling based upon the dlm package by Petris, Petrone, and Campagnoli, while trying some calculations and code for regime shift detection. One of the test cases … Continue reading

Posted in Bayesian, citizen science, climate change, climate data, climate disruption, dynamic linear models, floods, forecasting, Frequentist, global warming, icesheets, information theoretic statistics, Kalman filter, meteorology, open data, sea level rise, state-space models, statistics, time series | 1 Comment

“The storage necessity myth: how to choreograph high-renewables electricity systems”

(This was originally presented by CleanTechMedia.) Sounds like a great role for smart control systems. Flash COP21 won’t matter. Listen to Professor Tony Seba. (Use your browser Back button to return to this blog.) Excerpt: Clearly, though, many vested interests … Continue reading

Posted in adaptation, Anthropocene, Cape Wind, Carbon Tax, citizenship, clean disruption, climate change, climate disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, denial, dynamic linear models, dynamical systems, economics, efficiency, energy, energy reduction, energy utilities, engineering, fear uncertainty and doubt, forecasting, fossil fuel divestment, fossil fuels, global warming, Hyper Anthropocene, ignorance, investment in wind and solar energy, meteorology, microgrids, natural gas, obfuscating data, planning, politics, public utility commissions, PUCs, rationality, reasonableness, Sankey diagram, solar energy, solar power, SolarPV.tv, Stanford University, sustainability, the right to know, Tony Seba, University of California Berkeley, wind energy, wind power, zero carbon | 3 Comments

Thoughts on “Regime Shift?”

John Baez at The Azimuth Project opened a discussion on the recent paper by Reid, et al Philip C. Reid et al, Global impacts of the 1980s regime shift on the Earth’s climate and systems, Global Change Biology, 2015. I … Continue reading

Posted in Bayesian, changepoint detection, climate change, climate disruption, climate models, dynamic linear models, ecology, ensembles, environment, global warming, population biology, Rauch-Tung-Striebel, regime shifts, state-space models, stochastic algorithms, time series | Leave a comment

Is Earth Much More Sensitive to CO2 Than Thought?

“Take notice that carbon dioxide 50 million years ago may not have been as high as we once thought it was. We may reach that level in the next century, and so the climate change from that increase could be pretty severe, pretty dramatic. CO2 and other climate forcings may be more important for global warming than we realized.” Continue reading

Posted in Anthropocene, Carbon Cycle, carbon dioxide, climate, climate change, climate data, climate disruption, differential equations, diffusion processes, dynamic linear models, dynamical systems, environment, fossil fuels, generalized linear models, geophysics, global warming, Hyper Anthropocene, Principles of Planetary Climate, risk, science | Leave a comment

Southern Oscillation (SOI) correlated with Outgoing Longwave Radiation (OLR)

To the climate community this is nothing at all new, but I spotted these time series today and thought they would make a nice exhibit on how something people have direct control over, greenhouse gas emissions, affect a “teleconnection mechanism” … Continue reading

Posted in AMETSOC, bifurcations, carbon dioxide, climate, climate change, climate disruption, climate models, Dan Satterfield, differential equations, dynamic linear models, dynamical systems, ENSO, environment, forecasting, generalized linear models, geophysics, global warming, greenhouse gases, IPCC, Mathematica, mathematics, maths, meteorology, NCAR, NOAA, numerical software, oceanography, open data, physics, population biology, Principles of Planetary Climate, rationality, reasonableness, science, Spaceship Earth, state-space models, thermodynamics, time series | Leave a comment

On differential localization of tumors using relative concentrations of ctDNA. Part 1.

Like most mammalian tissue, tumors often produce shards of DNA as a byproduct of cell death and fracture. This circulating tumor DNA is being studied as a means of detecting tumors or their resurgence after treatment. (See also a Q&A … Continue reading

Posted in approximate Bayesian computation, Bayesian, Bayesian inversion, cardiovascular system, diffusion, dynamic linear models, eigenanalysis, engineering, forecasting, mathematics, maths, medicine, networks, prediction, spatial statistics, statistics, stochastic algorithms, stochastic search, wave equations | 3 Comments

On Changing Things

You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete. That’s from Buckminster Fuller, a fellow Unitarian.

Posted in adaptation, Anthropocene, bifurcations, bridge to nowhere, Buckminster Fuller, Cauchy distribution, clean disruption, climate disruption, demand-side solutions, destructive economic development, Disney, dynamic linear models, dynamical systems, Epcot, exponential growth, fossil fuel divestment, global warming, Hyper Anthropocene, physical materialism, planning, rationality, reasonableness, Spaceship Earth, stochastic algorithms | Leave a comment

Solar array with cloud predicting technology launched in WA

Australia’s first grid-connected solar power project with cloud predicting technology launched at Karratha Airport, WA, in bid to smooth solar supply. Source: Solar array with cloud predicting technology launched in WA

Posted in adaptation, Anthropocene, carbon dioxide, citizenship, civilization, clean disruption, climate, climate change, climate disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, dynamic linear models, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, forecasting, geophysics, global warming, Hyper Anthropocene, investment in wind and solar energy, Kalman filter, mathematics, maths, meteorology, microgrids, mitigation, NCAR, numerical software, optimization, physics, prediction, probabilistic programming, rationality, reasonableness, risk, science, solar power, stochastics, sustainability, time series | Leave a comment

Utilities for dummies: How they work and why that needs to change (from grist.org)

“Utilities are shielded by a force field of tedium.” “Solar panels could destroy U.S. utilities, according to U.S. utilities.” Utilities for dummies: How they work and why that needs to change“, a compact introduction, from grist.org. And there’s an additional … Continue reading

Posted in Anthropocene, bifurcations, bridge to nowhere, carbon dioxide, citizenship, civilization, clean disruption, climate, climate change, climate disruption, climate education, conservation, consumption, corruption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, disingenuity, dynamic linear models, dynamical systems, ecology, economics, education, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, exponential growth, finance, forecasting, fossil fuel divestment, fossil fuels, fracking, global warming, Hyper Anthropocene, ignorance, investing, investment in wind and solar energy, mathematics, maths, meteorology, methane, microgrids, natural gas, optimization, physics, pipelines, politics, prediction, public utility commissions, PUCs, rationality, reasonableness, risk, science, solar power, statistics, sustainability, taxes, temporal myopia, the right to know, time series, Tony Seba, wind power, zero carbon | 1 Comment

Solar installation progress, courtesy of MacSolarIndex.com

The MAC Solar Index tracks a set of solar manufacturing and installation companies. It is also the basis for the Guggenheim Investments “TAN” Exchange-Traded Fund (“ETF”, *). They recently published a progress report on global solar installations, which I wanted … Continue reading

Posted in adaptation, Anthropocene, carbon dioxide, citizen science, citizenship, civilization, clean disruption, climate, climate change, climate data, climate disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, dynamic linear models, dynamical systems, ecology, economics, efficiency, energy, energy reduction, environment, exponential growth, forecasting, fossil fuel divestment, fossil fuels, geophysics, global warming, Hyper Anthropocene, investing, investment in wind and solar energy, mathematics, mathematics education, maths, meteorology, microgrids, open data, optimization, physics, politics, prediction, rationality, reasonableness, risk, science, science education, solar power, sustainability, the right to know, time series, Tony Seba, wind power, zero carbon | Leave a comment

SCIENCE OF DOOM takes on assessing zero Carbon power and a zero Carbon grid

Updated, 2127 EDT, 10th August 2015 The blog, Science of Doom, has taken on a new thread discussing the technical feasibilities and problems associated with building out zero Carbon energy in the context of an electric grid. As such, it … Continue reading

Posted in adaptation, Anthropocene, clean disruption, climate data, climate disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, dynamic linear models, dynamical systems, economics, efficiency, energy, energy reduction, engineering, environment, exponential growth, forecasting, fossil fuel divestment, global warming, Hyper Anthropocene, investing, investment in wind and solar energy, microgrids, open data, optimization, prediction, rationality, reasonableness, risk, solar power, state-space models, stochastics, sustainability, the right to know, time series, wind power, Wordpress, zero carbon | 4 Comments

Comprehensive and compact tutorial on Petris’ DLM package in R; with an update about Helske’s KFAS

A blogger named Lalas produced on Quantitative Thoughts a very comprehensive and compact tutorial on the R package dlm by Petris. I use dlm a lot. Unfortunately, Lalas does not give details on how the SVD is used. They do … Continue reading

Posted in Bayes, Bayesian, dynamic linear models, dynamical systems, forecasting, Kalman filter, mathematics, maths, multivariate statistics, numerical software, open source scientific software, prediction, R, Rauch-Tung-Striebel, state-space models, statistics, stochastic algorithms, SVD, time series | 1 Comment