Category Archives: dependent data

Calculating Derivatives from Random Forests

Posted on 27 June 2020 by

(Edits to repair smudges, 2020-06-28, about 0945 EDT. Closing comment, 2020-06-30, 1450 EDT.) There are lots of ways of learning about mathematical constructs, even about actual machines. One way is to push them into a place of improbable application, asking … Continue reading →

Posted in bridge to somewhere, Calculus, dependent data, dynamic generalized linear models, dynamical systems, ensemble methods, ensemble models, filtering, forecasting, hierarchical clustering, linear regression, model-free forecasting, Monte Carlo Statistical Methods, non-mechanistic modeling, non-parametric model, non-parametric statistics, numerical algorithms, prediction, R statistical programming language, random forests, regression, sampling, splines, statistical learning, statistical series, statistics, time derivatives, time series | Leave a comment

Procrustes tangent distance is better than SNCD

Posted on 24 February 2019 by

I’ve written two posts here on using a Symmetrized Normalized Compression Divergence or SNCD for comparing time series. One introduced the SNCD and described its relationship to compression distance, and the other applied the SNCD to clustering days at a … Continue reading →

Posted in data science, dependent data, descriptive statistics, divergence measures, hydrology, Ian Dryden, information theoretic statistics, J.T.Kent, Kanti Mardia, non-parametric statistics, normalized compression divergence, quantitative ecology, R statistical programming language, spatial statistics, statistical series, time series | Leave a comment

Stream flow and P-splines: Using built-in estimates for smoothing

Posted on 28 January 2019 by

Mother Brook in Dedham Massachusetts was the first man-made canal in the United States. Dug in 1639, it connects the Charles River at Dedham, to the Neponset River in the Hyde Park section of Boston. It was originally an important … Continue reading →

Posted in American Statistical Association, citizen data, citizen science, Clausius-Clapeyron equation, Commonwealth of Massachusetts, cross-validation, data science, dependent data, descriptive statistics, dynamic linear models, empirical likelihood, environment, flooding, floods, Grant Foster, hydrology, likelihood-free, meteorological models, model-free forecasting, non-mechanistic modeling, non-parametric, non-parametric model, non-parametric statistics, numerical algorithms, precipitation, quantitative ecology, statistical dependence, statistical series, stream flow, Tamino, the bootstrap, time series, water vapor | 2 Comments

Less evidence for a global warming hiatus, and urging more use of Bayesian model averaging in climate science

Posted on 14 February 2018 by

(This post has been significantly updated midday 15th February 2018.) I’ve written about the supposed global warming hiatus of 2001-2014 before: “‘Overestimated global warming over the past 20 years’ (Fyfe, Gillett, Zwiers, 2013)”, 28 August 2013 “Warming Slowdown?”, Azimuth, Part … Continue reading →

Posted in American Statistical Association, Andrew Parnell, anomaly detection, Anthropocene, Bayesian, Bayesian model averaging, Berkeley Earth Surface Temperature project, BEST, climate change, David Spiegelhalter, dependent data, Dublin, GISTEMP, global warming, Grant Foster, HadCRUT4, hiatus, Hyper Anthropocene, JAGS, Markov Chain Monte Carlo, Martyn Plummer, Mathematics and Climate Research Network, MCMC, model-free forecasting, Niamh Cahill, Significance, statistics, Stefan Rahmstorf, Tamino | 2 Comments

What are the odds of net zero?

Posted on 1 January 2018 by

What’s the Question? A question was posed by a colleague a couple of months ago: What are the odds of a stock closing at the same price it opened? I found the question interesting, because, at first, it appeared to … Continue reading →

Posted in dependent data, evidence, financial series, investing, investments, model-free forecasting, numerical algorithms, state-space models, statistics, time series, trading | Leave a comment

`Evidence of a decline in electricity use by U.S. households’ (Prof Lucas Davis, U.C. Berkeley)

Posted on 8 May 2017 by

This is from a blog post by Professor Lucas Davis at his blog. In addition to the subject, that’s an interesting way of presenting a change over time I’ll need to think about: It seems the model could be used … Continue reading →

Posted in American Solar Energy Society, American Statistical Association, anomaly detection, Bloomberg New Energy Finance, BNEF, bridge to somewhere, convergent cross-mapping, decentralized electric power generation, decentralized energy, demand-side solutions, dependent data, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy utilities, engineering, evidence, green tech, local self reliance, Lucas Davis, marginal energy sources, Massachusetts Clean Energy Center, model-free forecasting, multivariate statistics, public utility commissions, rate of return regulation, statistics, Takens embedding theorem | Leave a comment

`Letter to Lamar Smith’

Posted on 28 March 2017 by

On Ed Hawkins’ blog. The Committee on Science, Space & Technology of the US House of Representatives conducts regular evidence hearings on various science topics. On Wednesday 29th March, there is a hearing on “Climate science: assumptions, policy implications, and … Continue reading →

Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, anemic data, anomaly detection, Anthropocene, Ben Santer, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, changepoint detection, climate, climate change, climate data, climate disruption, Climate Lab Book, climate zombies, dependent data, environment, fossil fuel divestment, geophysics, global warming, greenhouse gases, Humans have a lot to answer for, Hyper Anthropocene, leaving fossil fuels in the ground, meteorology, MIchael Mann, Our Children's Trust, physics, science, smoothing, statistical dependence, the right to be and act stupid, the right to know, the tragedy of our present civilization, the value of financial assets, time series | Leave a comment