Category Archives: model-free forecasting

Calculating Derivatives from Random Forests

Posted on 27 June 2020 by

(Comment on prediction intervals for random forests, and links to a paper.) (Edits to repair smudges, 2020-06-28, about 0945 EDT. Closing comment, 2020-06-30, 1450 EDT.) There are lots of ways of learning about mathematical constructs, even about actual machines. One … Continue reading →

Posted in bridge to somewhere, Calculus, dependent data, dynamic generalized linear models, dynamical systems, ensemble methods, ensemble models, filtering, forecasting, hierarchical clustering, linear regression, model-free forecasting, Monte Carlo Statistical Methods, non-mechanistic modeling, non-parametric model, non-parametric statistics, numerical algorithms, prediction, R statistical programming language, random forests, regression, sampling, splines, statistical learning, statistical series, statistics, time derivatives, time series | Leave a comment

What happens when time sampling density of a series matches its growth

Posted on 14 March 2020 by

This is the newly updated map of COVID-19 cases in the United States, updated, presumably, because of the new emphasis upon testing: How do we know this is the recent of recent testing? Look at the map of active cases: … Continue reading →

Posted in American Association for the Advancement of Science, American Statistical Association, anti-intellectualism, anti-science, climate denial, corruption, data science, data visualization, Donald Trump, dump Trump, epidemiology, experimental science, exponential growth, forecasting, Kalman filter, model-free forecasting, nonlinear systems, open data, penalized spline regression, population dynamics, sampling algorithms, statistical ecology, statistical models, statistical regression, statistical series, statistics, sustainability, the right to know, the stack of lies | 1 Comment

A response to a post on RealClimate

Posted on 26 June 2019 by

(Updated 2342 EDT, 28 June 2019.) This is a response to a post on RealClimate which primarily concerned economist Ross McKitrick’s op-ed in the Financial Post condemning the geophysical community for disregarding Roger Pielke, Jr’s arguments. Pielke, in that link, … Continue reading →

Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, Bayesian, climate change, ecology, Ecology Action, environment, evidence, experimental design, Frequentist, global warming, Hyper Anthropocene, machine learning, model comparison, model-free forecasting, multivariate statistics, science, science denier, statistical series, statistics, time series | Leave a comment

Stream flow and P-splines: Using built-in estimates for smoothing

Posted on 28 January 2019 by

Mother Brook in Dedham Massachusetts was the first man-made canal in the United States. Dug in 1639, it connects the Charles River at Dedham, to the Neponset River in the Hyde Park section of Boston. It was originally an important … Continue reading →

Posted in American Statistical Association, citizen data, citizen science, Clausius-Clapeyron equation, Commonwealth of Massachusetts, cross-validation, data science, dependent data, descriptive statistics, dynamic linear models, empirical likelihood, environment, flooding, floods, Grant Foster, hydrology, likelihood-free, meteorological models, model-free forecasting, non-mechanistic modeling, non-parametric, non-parametric model, non-parametric statistics, numerical algorithms, precipitation, quantitative ecology, statistical dependence, statistical series, stream flow, Tamino, the bootstrap, time series, water vapor | 2 Comments

Series, symmetrized Normalized Compressed Divergences and their logit transforms

Posted on 3 January 2019 by

(Major update on 11th January 2019. Minor update on 16th January 2019.) On comparing things The idea of a calculating a distance between series for various purposes has received scholarly attention for quite some time. The most common application is … Continue reading →

Posted in Akaike Information Criterion, bridge to somewhere, computation, content-free inference, data science, descriptive statistics, divergence measures, engineering, George Sughihara, information theoretic statistics, likelihood-free, machine learning, mathematics, model comparison, model-free forecasting, multivariate statistics, non-mechanistic modeling, non-parametric statistics, numerical algorithms, statistics, theoretical physics, thermodynamics, time series | 4 Comments
Posted on 5 July 2018 by

“One of the most interesting things about the MIP ensembles is that the mean of all the models generally has higher skill than any individual model.” We hold these truths to be self-evident, that all models are created equal, that … Continue reading →

Link | Posted on 5 July 2018 by | Leave a comment

Less evidence for a global warming hiatus, and urging more use of Bayesian model averaging in climate science

Posted on 14 February 2018 by

(This post has been significantly updated midday 15th February 2018.) I’ve written about the supposed global warming hiatus of 2001-2014 before: “‘Overestimated global warming over the past 20 years’ (Fyfe, Gillett, Zwiers, 2013)”, 28 August 2013 “Warming Slowdown?”, Azimuth, Part … Continue reading →

Posted in American Statistical Association, Andrew Parnell, anomaly detection, Anthropocene, Bayesian, Bayesian model averaging, Berkeley Earth Surface Temperature project, BEST, climate change, David Spiegelhalter, dependent data, Dublin, GISTEMP, global warming, Grant Foster, HadCRUT4, hiatus, Hyper Anthropocene, JAGS, Markov Chain Monte Carlo, Martyn Plummer, Mathematics and Climate Research Network, MCMC, model-free forecasting, Niamh Cahill, Significance, statistics, Stefan Rahmstorf, Tamino | 2 Comments

What are the odds of net zero?

Posted on 1 January 2018 by

What’s the Question? A question was posed by a colleague a couple of months ago: What are the odds of a stock closing at the same price it opened? I found the question interesting, because, at first, it appeared to … Continue reading →

Posted in dependent data, evidence, financial series, investing, investments, model-free forecasting, numerical algorithms, state-space models, statistics, time series, trading | Leave a comment

Eli on “Tom [Karl]’s trick and experimental design“

Posted on 11 November 2017 by

A very fine post at Eli’s blog for students of statistics, meteorology, and climate (like myself) titled: Tom’s trick and experimental design Excerpt: This and the graph from Menne at the top shows that Karl’s trick is working. Although we … Continue reading →

Quote | Posted on 11 November 2017 by | Leave a comment

Statements by the Ecological Society of America on the proposed U.S. exit from the Paris Agreement, and on Climate Change

Posted on 15 June 2017 by

By withdrawing from the Paris Agreement on climate change, the United States is abdicating its role as the world leader in using science-based information to inform policy. Business, political, and scientific leaders the world over are condemning the decision. More … Continue reading →

Posted in adaptation, American Association for the Advancement of Science, American Statistical Association, Anthropocene, argoecology, Carl Safina, climate change, climate disruption, complex systems, ecological services, ecology, Ecology Action, environment, global warming, Hyper Anthropocene, marine biology, mesh models, model-free forecasting, population biology, population dynamics, quantitative biology, quantitative ecology, science, Science magazine, Spaceship Earth, sustainability, Takens embedding theorem, the tragedy of our present civilization, the value of financial assets, tragedy of the horizon, Wordpress, zero carbon | Leave a comment

`Evidence of a decline in electricity use by U.S. households’ (Prof Lucas Davis, U.C. Berkeley)

Posted on 8 May 2017 by

This is from a blog post by Professor Lucas Davis at his blog. In addition to the subject, that’s an interesting way of presenting a change over time I’ll need to think about: It seems the model could be used … Continue reading →

Posted in American Solar Energy Society, American Statistical Association, anomaly detection, Bloomberg New Energy Finance, BNEF, bridge to somewhere, convergent cross-mapping, decentralized electric power generation, decentralized energy, demand-side solutions, dependent data, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy utilities, engineering, evidence, green tech, local self reliance, Lucas Davis, marginal energy sources, Massachusetts Clean Energy Center, model-free forecasting, multivariate statistics, public utility commissions, rate of return regulation, statistics, Takens embedding theorem | Leave a comment

Liang, information flows, causation, and convergent cross-mapping

Posted on 2 May 2017 by

Someone recommended the work of Liang recently in connection with causation and attribution studies, and their application to CO2 and climate change. Liang’s work is related to information flows and transfer entropies. As far as I know, the definitive work … Continue reading →

Posted in Akaike Information Criterion, American Association for the Advancement of Science, Anthropocene, attribution, carbon dioxide, climate, climate change, climate disruption, complex systems, convergent cross-mapping, ecology, Egbert van Nes, Ethan Deyle, Floris Takens, George Sughihara, global warming, Hao Ye, Hyper Anthropocene, information theoretic statistics, Lenny Smith, model-free forecasting, nonlinear systems, physics, statistics, Takens embedding theorem, theoretical physics, Timothy Lenton, Victor Brovkin | 1 Comment

Just because the data lies sometimes doesn’t mean it’s okay to censor it

Posted on 1 May 2017 by

Or, there’s no such thing as an outlier … Eli put up a post titled “The Data Lies. The Crisis in Observational Science and the Virtue of Strong Theory” at his lagomorph blog. Think of it: Data lying. Obviously this … Continue reading →

Posted in Akaike Information Criterion, American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, Anthropocene, Bayes, Bayesian, climate, climate change, climate models, data science, dynamical systems, ecology, Eli Rabett, environment, Ethan Deyle, George Sughihara, Hao Ye, Hyper Anthropocene, information theoretic statistics, IPCC, Kalman filter, kriging, Lenny Smith, maximum likelihood, model comparison, model-free forecasting, physics, quantitative ecology, random walk processes, random walks, science, smart data, state-space models, statistics, Takens embedding theorem, the right to know, Timothy Lenton, Victor Brovkin | 1 Comment

“Hurricanes, Sea Level, and Baloney” (from Tamino)

Posted on 28 October 2016 by

Originally posted on Open Mind:
WUWT has a post in which Neil Frank proclaims that Hillary Clinton is no hurricane expert but he is. (Frank’s post was originally published on The Daily Caller, but was reprinted on WUWT with permission.)…

Posted in American Meteorological Association, AMETSOC, climate, climate change, climate disruption, denial, environment, global warming, Hyper Anthropocene, meteorological models, meteorology, model-free forecasting, open data, science denier | Leave a comment

Cathy O’Neil’s WEAPONS OF MATH DESTRUCTION: A Review

Posted on 23 October 2016 by

(Revised and updated Monday, 24th October 2016.) Weapons of Math Destruction, Cathy O’Neil, published by Crown Random House, 2016. This is a thoughtful and very approachable introduction and review to the societal and personal consequences of data mining, data science, … Continue reading →

Posted in citizen data, citizen science, citizenship, civilization, compassion, complex systems, criminal justice, Daniel Kahneman, data science, deep recurrent neural networks, destructive economic development, economics, education, engineering, ethics, Google, ignorance, Joseph Schumpeter, life purpose, machine learning, Mathbabe, mathematics, mathematics education, maths, model comparison, model-free forecasting, numerical analysis, numerical software, open data, optimization, organizational failures, planning, politics, prediction, prediction markets, privacy, rationality, reason, reasonableness, risk, silly tech devices, smart data, sociology, Techno Utopias, testing, the value of financial assets, transparency | Leave a comment

“All models are wrong. Some models are useful.” — George Box

Posted on 17 October 2016 by

(Image courtesy of the Damien Garcia.) As a statistician and quant, I’ve thought hard about that oft-cited Boxism. I’m not sure I agree. It’s not that there is such a thing as a perfect model, or correct model, whatever in … Continue reading →

Posted in abstraction, American Association for the Advancement of Science, astronomy, astrophysics, mathematics, model-free forecasting, numerics, perceptions, physical materialism, physics, rationality, reason, reasonableness, science, spatial statistics, splines, statistics, the right to know, theoretical physics, time series | Leave a comment

Carbon Sinks in Crisis — It Looks Like the World’s Largest Rainforest is Starting to Bleed Greenhouse Gasses

Posted on 16 August 2016 by

Originally posted on robertscribbler:
Back in 2005, and again in 2010, the vast Amazon rainforest, which has been aptly described as the world’s lungs, briefly lost its ability to take in atmospheric carbon dioxide. Its drought-stressed trees were not growing…

Posted in bifurcations, carbon dioxide, carbon dioxide sequestration, changepoint detection, climate, climate change, climate disruption, disruption, dynamical systems, environment, exponential growth, fossil fuels, geophysics, global warming, IPCC, Lévy flights, Lorenz, Minsky moment, model-free forecasting, physics, population biology, population dynamics, Principles of Planetary Climate, quantitative biology, quantitative ecology, random walk processes, Ray Pierrehumbert, reason, reasonableness, regime shifts, risk, Stefan Rahmstorf, the right to be and act stupid, the tragedy of our present civilization, UU Humanists | 2 Comments

Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories

Posted on 11 August 2016 by

(Slight update, 28th June 2020.) I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random … Continue reading →

Posted in American Meteorological Association, American Statistical Association, AMETSOC, Arthur Charpentier, atmosphere, diffusion, diffusion processes, dynamic linear models, dynamical systems, environment, geophysics, hurricanes, Kalman filter, Kerry Emanuel, Lévy flights, Lorenz, Markov chain random fields, mathematics, mathematics education, maths, MCMC, mesh models, meteorological models, meteorology, model-free forecasting, Monte Carlo Statistical Methods, numerical analysis, numerical software, oceanography, open data, open source scientific software, physics, random walk processes, random walks, science, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, time series | 1 Comment

“Stochastic Parameterization: Towards a new view of weather and climate models”

Posted on 24 July 2016 by

Judith Berner, Ulrich Achatz, Lauriane Batté, Lisa Bengtsson, Alvaro De La Cámara, Hannah M. Christensen, Matteo Colangeli, Danielle R. B. Coleman, Daan Crommelin, Stamen I. Dolaptchiev, Christian L.E. Franzke, Petra Friederichs, Peter Imkeller, Heikki Järvinen, Stephan Juricke, Vassili Kitsios, François … Continue reading →

Posted in biology, climate models, complex systems, convergent cross-mapping, data science, dynamical systems, ecology, Ethan Deyle, Floris Takens, George Sughihara, Hao Ye, likelihood-free, Lorenz, mathematics, meteorological models, model-free forecasting, physics, population biology, population dynamics, quantitative biology, quantitative ecology, Scripps Institution of Oceanography, state-space models, statistical dependence, statistics, stochastic algorithms, stochastic search, stochastics, Takens embedding theorem, time series, Victor Brovkin | 4 Comments