Category Archives: prediction

Phase Plane plots of COVID-19 deaths with uncertainties

I. Introduction. It’s time to fulfill the promise made in “Phase plane plots of COVID-19 deaths“, a blog post from 2nd May 2020, and produce the same with uncertainty clouds about the functional trajectories(*). To begin, here are some assumptions … Continue reading

Posted in American Statistical Association, Andrew Harvey, anomaly detection, count data regression, COVID-19, dependent data, dlm package, Durbin and Koopman, dynamic linear models, epidemiology, filtering, forecasting, Kalman filter, LaTeX, model-free forecasting, Monte Carlo Statistical Methods, numerical algorithms, numerical linear algebra, population biology, population dynamics, prediction, R, R statistical programming language, regression, statistical learning, stochastic algorithms | Tagged | Leave a comment

Calculating Derivatives from Random Forests

(Comment on prediction intervals for random forests, and links to a paper.) (Edits to repair smudges, 2020-06-28, about 0945 EDT. Closing comment, 2020-06-30, 1450 EDT.) There are lots of ways of learning about mathematical constructs, even about actual machines. One … Continue reading

Posted in bridge to somewhere, Calculus, dependent data, dynamic generalized linear models, dynamical systems, ensemble methods, ensemble models, filtering, forecasting, hierarchical clustering, linear regression, model-free forecasting, Monte Carlo Statistical Methods, non-mechanistic modeling, non-parametric model, non-parametric statistics, numerical algorithms, prediction, R statistical programming language, random forests, regression, sampling, splines, statistical learning, statistical series, statistics, time derivatives, time series | Leave a comment

“Code for causal inference: Interested in astronomical applications”

via Code for causal inference: Interested in astronomical applications From Professor Ewan Cameron at his Another Astrostatistics Blog.

Posted in American Association for the Advancement of Science, American Statistical Association, astronomy, astrostatistics, causal inference, causation, counterfactuals, epidemiology, experimental design, experimental science, multivariate statistics, prediction, propensity scoring, quantitative biology, quantitative ecology, reproducible research, rhetorical mathematics, rhetorical science, rhetorical statistics, science, statistical ecology, statistical models, statistical regression, statistics | Leave a comment

A look at an electricity consumption series using SNCDs for clustering

(Slightly amended with code and data link, 12th January 2019.) Prediction of electrical load demand or, in other words, electrical energy consumption is important for the proper operation of electrical grids, at all scales. RTOs and ISOs forecast demand based … Continue reading

Posted in American Statistical Association, consumption, data streams, decentralized electric power generation, dendrogram, divergence measures, efficiency, electricity, electricity markets, energy efficiency, energy utilities, ensembles, evidence, forecasting, grid defection, hierarchical clustering, hydrology, ILSR, information theoretic statistics, local self reliance, Massachusetts, microgrids, NCD, normalized compression divergence, numerical software, open data, prediction, rate of return regulation, Sankey diagram, SNCD, statistical dependence, statistical series, statistics, sustainability, symmetric normalized compression divergence, time series | 2 Comments

Sea-level report cards, contingency upon model character, and ensemble methods

Done by the Virginia Institute of Marine Science, new sea-level report cards offer a look at current sea-level rise rates, and projections. What’s interesting to me is making the projections conditional upon the character of the model used to project. … Continue reading

Posted in adaptation, American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, anomaly detection, Bayesian model averaging, changepoint detection, climate disruption, climate economics, climate education, coastal communities, coasts, dynamical systems, ensemble methods, ensemble models, flooding, geophysics, global warming, Grant Foster, Hyper Anthropocene, ice sheet dynamics, icesheets, living shorelines, meteorological models, nonlinear systems, prediction, sea level rise, shorelines, Skeptical Science, spaghetti plots, temporal myopia, the right to be and act stupid, the right to know, the value of financial assets, tragedy of the horizon | Leave a comment

Cathy O’Neil’s WEAPONS OF MATH DESTRUCTION: A Review

(Revised and updated Monday, 24th October 2016.) Weapons of Math Destruction, Cathy O’Neil, published by Crown Random House, 2016. This is a thoughtful and very approachable introduction and review to the societal and personal consequences of data mining, data science, … Continue reading

Posted in citizen data, citizen science, citizenship, civilization, compassion, complex systems, criminal justice, Daniel Kahneman, data science, deep recurrent neural networks, destructive economic development, economics, education, engineering, ethics, Google, ignorance, Joseph Schumpeter, life purpose, machine learning, Mathbabe, mathematics, mathematics education, maths, model comparison, model-free forecasting, numerical analysis, numerical software, open data, optimization, organizational failures, planning, politics, prediction, prediction markets, privacy, rationality, reason, reasonableness, risk, silly tech devices, smart data, sociology, Techno Utopias, testing, the value of financial assets, transparency | Leave a comment

“Holy crap – an actual book!”

Originally posted on mathbabe:
Yo, everyone! The final version of my book now exists, and I have exactly one copy! Here’s my editor, Amanda Cook, holding it yesterday when we met for beers: Here’s my son holding it: He’s offered…

Posted in American Association for the Advancement of Science, Buckminster Fuller, business, citizen science, citizenship, civilization, complex systems, confirmation bias, data science, data streams, deep recurrent neural networks, denial, economics, education, engineering, ethics, evidence, Internet, investing, life purpose, machine learning, mathematical publishing, mathematics, mathematics education, maths, moral leadership, multivariate statistics, numerical software, numerics, obfuscating data, organizational failures, politics, population biology, prediction, prediction markets, privacy, quantitative biology, quantitative ecology, rationality, reason, reasonableness, rhetoric, risk, Schnabel census, smart data, sociology, statistical dependence, statistics, the right to be and act stupid, the right to know, the value of financial assets, transparency, UU Humanists | Leave a comment

On Munshi mush

(Slightly updated on 2016-06-11.) Professor Emeritus Jamal Munshi of Sonoma State University has papers recently cited in science denier circles as evidence that the conventional associations between mean global surface temperature and cumulative carbon emissions are, well, bunk, due to … Continue reading

Posted in Bayes, Bayesian, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, cat1, climate, climate change, climate data, climate education, climate models, convergent cross-mapping, dynamic linear models, ecology, ENSO, environment, Ethan Deyle, evidence, geophysics, George Sughihara, global warming, greenhouse gases, information theoretic statistics, Kalman filter, mathematics, maths, meteorology, model comparison, NOAA, oceanography, prediction, state-space models, statistics, Takens embedding theorem, Techno Utopias, the right to know, theoretical physics, time series, zero carbon | 1 Comment

Where’s NY-REV today?

This is a video and link from The Rocky Mountain Institute about New York State’s Reforming the Energy Visions or “REV”. I have written about REV before.

Posted in adaptation, Anthropocene, Bloomberg New Energy Finance, BNEF, bridge to somewhere, Buckminster Fuller, citizenship, civilization, clean disruption, climate change, climate disruption, conservation, consumption, corporate litigation on damage from fossil fuel emissions, decentralized electric power generation, decentralized energy, demand-side solutions, distributed generation, Ecology Action, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy storage, energy utilities, feed-in tariff, fossil fuel divestment, global warming, Green Tea Coalition, greenhouse gases, grid defection, Hermann Scheer, Hyper Anthropocene, investment in wind and solar energy, ISO-NE, leaving fossil fuels in the ground, Life Cycle Assessment, local generation, long-term contract for differences, marginal energy sources, Mark Jacobson, microgrids, mitigation, optimization, planning, prediction, public utility commissions, PUCs, rate of return regulation, rationality, reasonableness, regulatory capture, RevoluSun, risk, Sankey diagram, solar domination, solar energy, Solar Freakin' Roadways, solar power, SolarPV.tv, Spaceship Earth, stranded assets, sustainability, the energy of the people, the green century, the value of financial assets, Tony Seba, utility company death spiral, wind energy, wind power, zero carbon | Leave a comment

Boston, are you ready?

Yeah, how about warming up the seas a bit more by building pipelines, buying into more explosive methane (*), and encouraging fracked gas people to export? What could it hurt? There are many alternatives, most sketched here on this blog. … Continue reading

Posted in adaptation, AMETSOC, Anthropocene, bollocks, Boston, bridge to nowhere, carbon dioxide, Carbon Worshipers, climate change, climate disruption, corporate litigation on damage from fossil fuel emissions, corruption, disingenuity, ecology, evidence, false advertising, floods, Florida, fossil fuel divestment, fossil fuels, fracking, geophysics, global warming, greenwashing, Hyper Anthropocene, MA, Massachusetts, methane, mitigation, natural gas, NOAA, nor'easters, physics, pipelines, prediction, rate of return regulation, rationality, reasonableness, regulatory capture, science, sea level rise, sustainability, the right to be and act stupid, the right to know, the tragedy of our present civilization, utility company death spiral | 1 Comment

“What’s the deal with sea level rise?”

Update: 20th June 2016 “Rising seas: Should I say or should I go?“, by Delavane Diaz WunderBlog.

Posted in adaptation, AMOC, Antarctica, Anthropocene, Arctic, Carbon Worshipers, civilization, dynamical systems, Eaarth, geophysics, glaciers, glaciology, global warming, Guy McPherson, Hyper Anthropocene, ice sheet dynamics, James Hansen, oceanography, physics, planning, prediction, Principles of Planetary Climate, regime shifts, risk, science, sea level rise, the tragedy of our present civilization, thermohaline circulation, Wally Broecker | Leave a comment

HadCRUT4 and GISTEMP series filtered and estimated with simple RTS model

Happy Vernal Equinox! This post has been updated today with some of the equations which correspond to the models. An assessment of whether or not there was a meaningful slowdown or “hiatus” in global warming, was recently discussed by Tamino … Continue reading

Posted in AMETSOC, anemic data, Bayesian, boosting, bridge to somewhere, cat1, changepoint detection, climate, climate change, climate data, climate disruption, climate models, complex systems, computation, data science, dynamical systems, geophysics, George Sughihara, global warming, hiatus, information theoretic statistics, machine learning, maths, meteorology, MIchael Mann, multivariate statistics, physics, prediction, Principles of Planetary Climate, rationality, reasonableness, regime shifts, sea level rise, time series | 5 Comments

Causal Diagrams

Like Sankey diagrams, causal diagrams are a useful tool to assess and communicate complicated systems and their intrarelationships: It’s possible to use these for analysis and prescription: Here is the (promised) presentation on reenforcing loops: So how can these techniques … Continue reading

Posted in adaptation, bridge to nowhere, Carbon Cycle, carbon dioxide, carbon dioxide sequestration, Carbon Tax, Carbon Worshipers, causal diagrams, clean disruption, climate, climate change, climate disruption, climate education, climate models, demand-side solutions, differential equations, dynamical systems, ecology, economics, energy utilities, environment, exponential growth, fossil fuel divestment, fossil fuels, global warming, greenhouse gases, greenwashing, Hyper Anthropocene, mathematics, mathematics education, maths, methane, mitigation, natural gas, planning, prediction, rationality, reasonableness, recycling, Sankey diagram, sustainability, the right to know, zero carbon | Leave a comment

All I do is complain, complain …

I was reviewing a presentation given as part of a short course in the machine learning genre today, and happened across the following two bullets, under the heading “Strictly Stationary Processes”: Predicting a time series is possible if and only … Continue reading

Posted in bifurcations, chaos, citizen science, convergent cross-mapping, dynamic linear models, dynamical systems, engineering, Floris Takens, generalized linear models, geophysics, George Sughihara, ignorance, Lenny Smith, Lorenz, mathematics, maths, meteorology, prediction, probability, rationality, reasonableness, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, the right to know, time series | 1 Comment

Wind and Solar are Cheaper than Fossil Fuels and Nuclear Right Now

… and that’s based upon levelized cost of energy, without subsidies! See a summary of Lazard’s report, the key chart below: (Click on image to see a larger version. Click on your browser’s Back button to return to blog.) the … Continue reading

Posted in bifurcations, Cape Wind, Carbon Worshipers, clean disruption, conservation, consumption, decentralized electric power generation, decentralized energy, destructive economic development, economics, efficiency, EIA, energy, energy reduction, energy utilities, engineering, fossil fuel divestment, fossil fuels, investing, investment in wind and solar energy, methane, microgrids, natural gas, nuclear power, open data, pipelines, politics, prediction, public utility commissions, PUCs, Sankey diagram, solar energy, solar power, SolarPV.tv, sustainability, temporal myopia, the value of financial assets, Tony Seba, wind energy, wind power | Leave a comment

sonnenCommunity connects households and makes conventional electricity suppliers obsolete through self-generated power

Sonnenbatterie presents the sonnenCommunity; members can generate their own power, store it and share surpluses online with friends or each other. The sonnenCommunity completely replaces traditional power companies and will soon be available to every household in Germany. – sonnenCommunity … Continue reading

Posted in Anthropocene, Bill Gates, carbon dioxide, Carbon Worshipers, clean disruption, climate disruption, conservation, consumption, COP21, decentralized electric power generation, decentralized energy, demand-side solutions, economics, energy, energy reduction, energy utilities, engineering, environment, fossil fuel divestment, global warming, Hyper Anthropocene, investment in wind and solar energy, mesh models, meteorology, microgrids, planning, prediction, solar energy, solar power, SolarPV.tv, sustainability, Tony Seba, wind energy, zero carbon | Leave a comment

What the future of energy everywhere looks like

What will the energy landscape look like after utility companies are either dead, dying, or revert to a tiny portion of their territory? Silicon Valley CCE Partnership gives us all a clue. It’s been described in the San Francisco Chronicle, … Continue reading

Posted in adaptation, Anthropocene, capricious gods, chance, citizenship, civilization, clean disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, dynamical systems, economics, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, forecasting, fossil fuel divestment, geophysics, global warming, Hyper Anthropocene, investment in wind and solar energy, living shorelines, mesh models, meteorology, microgrids, mitigation, obfuscating data, oceanography, physical materialism, physics, pipelines, planning, politics, prediction, probabilistic programming, public utility commissions, PUCs, quantum, reasonableness, reproducible research, risk, Sankey diagram, science, sea level rise, selfishness, solar energy, solar power, SolarPV.tv, Spaceship Earth, statistics, stochastic algorithms, stochastics, Svante Arrhenius, taxes, temporal myopia, the right to know, the value of financial assets, transparency, UU Humanists, WHOI, wind energy, wind power, zero carbon | Leave a comment

New York State eliminates caps on solar net metering

In a decision by the New York State Public Service Commission on 16th October 2015, ceilings on the interconnection of distributed generation by sources like residential solar PV, sometimes called “caps”, were indefinitely suspended for all utility companies in the … Continue reading

Posted in citizenship, civilization, clean disruption, climate change, climate education, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, economics, efficiency, energy, energy utilities, environment, global warming, Hyper Anthropocene, investing, investment in wind and solar energy, meteorology, microgrids, prediction, public utility commissions, PUCs, rationality, reasonableness, solar energy, solar power, SolarPV.tv, sustainability, Tony Seba, wind energy, wind power, zero carbon | Leave a comment

Kerry Emanuel and Svante Arrhenius, hurricanes and climate disruption: A 2015 update

Posted in Anthropocene, carbon dioxide, climate, climate change, climate disruption, geophysics, global warming, hurricanes, Kerry Emanuel, meteorology, oceanography, physics, prediction, rationality, reasonableness, risk, science, sea level rise, sustainability, Svante Arrhenius | Leave a comment

New Study Projects That Melting of Antarctic Ice Shelves Will Intensify

New research published today projects a doubling of surface melting of Antarctic ice shelves by 2050 and by 2100 may surpass intensities associated with ice shelf collapse, if greenhouse gas emissions from fossil fuel consumption continues at the present rate. … Continue reading

Posted in Anthropocene, capricious gods, carbon dioxide, civilization, climate, climate disruption, ecology, exponential growth, geophysics, global warming, Hyper Anthropocene, icesheets, James Hansen, meteorology, NASA, NOAA, oceanography, physics, prediction, risk, science, sea level rise, sustainability, temporal myopia, WAIS, WHOI | Comments Off on New Study Projects That Melting of Antarctic Ice Shelves Will Intensify

On differential localization of tumors using relative concentrations of ctDNA. Part 1.

Like most mammalian tissue, tumors often produce shards of DNA as a byproduct of cell death and fracture. This circulating tumor DNA is being studied as a means of detecting tumors or their resurgence after treatment. (See also a Q&A … Continue reading

Posted in approximate Bayesian computation, Bayesian, Bayesian inversion, cardiovascular system, diffusion, dynamic linear models, eigenanalysis, engineering, forecasting, mathematics, maths, medicine, networks, prediction, spatial statistics, statistics, stochastic algorithms, stochastic search, wave equations | 3 Comments

How nice it is that Nature and probability bend to developers whims!

As I have mentioned before, it’s so nice that Nature and probability bend to the whims of property developers and their Town Fathers, with the willing participation of the Federal Emergency Management Agency (FEMA). If you had property at risk … Continue reading

Posted in capricious gods, chance, citizenship, climate data, conservation, denial, ecology, engineering, environment, ethics, games of chance, ignorance, living shorelines, mathematics, meteorology, obfuscating data, planning, politics, precipitation, prediction, probability, rationality, reasonableness, risk, spatial statistics, University Station, Westwood | 1 Comment

News Flash: “Warmer climate on the Earth may be due to more Carbon Dioxide in the air”

28th October 1956, The New York Times. Andy Dessler at TAMU Physics Department seminar, 24the September 2015.

Posted in Anthropocene, astrophysics, carbon dioxide, citizenship, climate disruption, compassion, denial, disingenuity, ecology, environment, forecasting, geophysics, global warming, Hyper Anthropocene, IPCC, physics, prediction, Principles of Planetary Climate, rationality, Ray Pierrehumbert, reasonableness, science, zero carbon | Leave a comment

“Too late to prevent climate change: Here’s how we adapt” (Alice Bows-Larkin)

Here’s how we adapt.

Posted in adaptation, Anthropocene, astrophysics, bridge to nowhere, capricious gods, carbon dioxide, chance, citizenship, civilization, climate, climate change, climate disruption, climate education, ecology, economics, environment, ethics, forecasting, fossil fuels, games of chance, geophysics, global warming, Hyper Anthropocene, ignorance, mathematics, maths, meteorology, mitigation, oceanography, physics, planning, prediction, probability, rationality, reasonableness, risk, science, statistics, sustainability, zero carbon | Leave a comment

Incredible Rainfall In South Carolina, and Yes Climate Played A Role – Dan’s Wild Wild Science Journal – AGU Blogosphere

Make no mistake, this was a flood event unlike any other in South Carolina and while Hurricane Joaquin never hit the coast, it holds a smoking gun. This flood was the result of several factors, an … Source: Incredible Rainfall … Continue reading

Posted in adaptation, Anthropocene, carbon dioxide, citizenship, civilization, climate change, climate disruption, Dan Satterfield, environment, forecasting, geophysics, global warming, hurricanes, Hyper Anthropocene, ignorance, IPCC, James Hansen, maths, meteorology, mitigation, NOAA, oceanography, physics, prediction, Principles of Planetary Climate, probability, rationality, Ray Pierrehumbert, reasonableness, science, science education, spatial statistics, statistics, sustainability, Tamino, the right to know, zero carbon | Leave a comment

Questions About El Nino Answered – Dan’s Wild Wild Science Journal – AGU Blogosphere

Source: Questions About El Nino Answered – Dan’s Wild Wild Science Journal – AGU Blogosphere

Posted in bifurcations, capricious gods, chance, climate change, Dan Satterfield, ensembles, ENSO, games of chance, maths, meteorology, NOAA, nor'easters, oceanography, precipitation, prediction, rationality, reasonableness, risk, science, spatial statistics | Leave a comment

Solar array with cloud predicting technology launched in WA

Australia’s first grid-connected solar power project with cloud predicting technology launched at Karratha Airport, WA, in bid to smooth solar supply. Source: Solar array with cloud predicting technology launched in WA

Posted in adaptation, Anthropocene, carbon dioxide, citizenship, civilization, clean disruption, climate, climate change, climate disruption, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, dynamic linear models, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, forecasting, geophysics, global warming, Hyper Anthropocene, investment in wind and solar energy, Kalman filter, mathematics, maths, meteorology, microgrids, mitigation, NCAR, numerical software, optimization, physics, prediction, probabilistic programming, rationality, reasonableness, risk, science, solar power, stochastics, sustainability, time series | Leave a comment

Locally generated solar PV will destroy America’s electric utilities: Barclays

We fully expect utilities and regulators to make a good faith effort to preserve the status quo “regulatory compact,” whereby the monopoly utility provides a safe and reliable service and regulators allow it to earn a reasonable low-risk return. However, … Continue reading

Posted in adaptation, Anthropocene, bridge to nowhere, Cape Wind, carbon dioxide, citizenship, clean disruption, climate, climate change, climate disruption, compassion, conservation, consumption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, dynamical systems, economics, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, exponential growth, forecasting, fossil fuel divestment, fossil fuels, geophysics, global warming, Hyper Anthropocene, investing, investment in wind and solar energy, meteorology, microgrids, physics, prediction, public utility commissions, PUCs, rationality, reasonableness, risk, solar power, statistics, Tony Seba, wind power, zero carbon | Leave a comment

“Allocating a 2° C cumulative carbon budget to countries”: Gignac and Matthews

Abstract Recent estimates of the global carbon budget, or allowable cumulative CO2 emissions consistent with a given level of climate warming, have the potential to inform climate mitigation policy discussions aimed at maintaining global temperatures below 2° C. This raises … Continue reading

Posted in adaptation, Anthropocene, Boston Ethical Society, bridge to nowhere, carbon dioxide, Carbon Tax, chance, chemistry, citizen science, citizenship, civilization, clean disruption, climate, climate change, climate disruption, climate education, climate justice, compassion, conservation, consumption, decentralized energy, demand-side solutions, destructive economic development, ecology, education, energy, energy reduction, environment, ethics, forecasting, fossil fuel divestment, fossil fuels, geophysics, global warming, humanism, Hyper Anthropocene, ignorance, investment in wind and solar energy, IPCC, meteorology, mitigation, open source scientific software, physical materialism, physics, population biology, prediction, rationality, reasonableness, risk, science, science education, scientific publishing, sociology, solar power, sustainability, temporal myopia, the right to know, time series, UNFCCC, UU Humanists, wind power | Leave a comment

Utilities for dummies: How they work and why that needs to change (from grist.org)

“Utilities are shielded by a force field of tedium.” “Solar panels could destroy U.S. utilities, according to U.S. utilities.” Utilities for dummies: How they work and why that needs to change“, a compact introduction, from grist.org. And there’s an additional … Continue reading

Posted in Anthropocene, bifurcations, bridge to nowhere, carbon dioxide, citizenship, civilization, clean disruption, climate, climate change, climate disruption, climate education, conservation, consumption, corruption, decentralized electric power generation, decentralized energy, demand-side solutions, destructive economic development, disingenuity, dynamic linear models, dynamical systems, ecology, economics, education, efficiency, energy, energy reduction, energy utilities, engineering, environment, ethics, exponential growth, finance, forecasting, fossil fuel divestment, fossil fuels, fracking, global warming, Hyper Anthropocene, ignorance, investing, investment in wind and solar energy, mathematics, maths, meteorology, methane, microgrids, natural gas, optimization, physics, pipelines, politics, prediction, public utility commissions, PUCs, rationality, reasonableness, risk, science, solar power, statistics, sustainability, taxes, temporal myopia, the right to know, time series, Tony Seba, wind power, zero carbon | 1 Comment