Distributed Solar: The Democratizaton of Energy
Category Archives: statistical dependence
Stream flow and P-splines: Using built-in estimates for smoothing
Mother Brook in Dedham Massachusetts was the first man-made canal in the United States. Dug in 1639, it connects the Charles River at Dedham, to the Neponset River in the Hyde Park section of Boston. It was originally an important … Continue reading
Posted in American Statistical Association, citizen data, citizen science, Clausius-Clapeyron equation, Commonwealth of Massachusetts, cross-validation, data science, dependent data, descriptive statistics, dynamic linear models, empirical likelihood, environment, flooding, floods, Grant Foster, hydrology, likelihood-free, meteorological models, model-free forecasting, non-mechanistic modeling, non-parametric, non-parametric model, non-parametric statistics, numerical algorithms, precipitation, quantitative ecology, statistical dependence, statistical series, stream flow, Tamino, the bootstrap, time series, water vapor
Leave a comment
A look at an electricity consumption series using SNCDs for clustering
(Slightly amended with code and data link, 12th January 2019.) Prediction of electrical load demand or, in other words, electrical energy consumption is important for the proper operation of electrical grids, at all scales. RTOs and ISOs forecast demand based … Continue reading
Posted in American Statistical Association, consumption, data streams, decentralized electric power generation, dendrogram, divergence measures, efficiency, electricity, electricity markets, energy efficiency, energy utilities, ensembles, evidence, forecasting, grid defection, hierarchical clustering, hydrology, ILSR, information theoretic statistics, local self reliance, Massachusetts, microgrids, NCD, normalized compression divergence, numerical software, open data, prediction, rate of return regulation, Sankey diagram, SNCD, statistical dependence, statistical series, statistics, sustainability, symmetric normalized compression divergence, time series
Leave a comment
The Rule of 135
From SingingBanana.
`Letter to Lamar Smith’
On Ed Hawkins’ blog. The Committee on Science, Space & Technology of the US House of Representatives conducts regular evidence hearings on various science topics. On Wednesday 29th March, there is a hearing on “Climate science: assumptions, policy implications, and … Continue reading
Posted in American Association for the Advancement of Science, American Meteorological Association, American Statistical Association, AMETSOC, anemic data, anomaly detection, Anthropocene, Ben Santer, Berkeley Earth Surface Temperature project, BEST, carbon dioxide, changepoint detection, climate, climate change, climate data, climate disruption, Climate Lab Book, climate zombies, dependent data, environment, fossil fuel divestment, geophysics, global warming, greenhouse gases, Humans have a lot to answer for, Hyper Anthropocene, leaving fossil fuels in the ground, meteorology, MIchael Mann, Our Children's Trust, physics, science, smoothing, statistical dependence, the right to be and act stupid, the right to know, the tragedy of our present civilization, the value of financial assets, time series
Leave a comment
Why smooth?
I’ve encountered a number of blog posts this week which seem not to understand the Bias-Variance Tradeoff in regard to Mean-Squared-Error. These arose in connection with smoothing splines, which I was studying in connection with multivariate adaptive regression splines, that … Continue reading
Posted in Akaike Information Criterion, American Statistical Association, Antarctica, carbon dioxide, climate change, denial, global warming, information theoretic statistics, likelihood-free, multivariate adaptive regression splines, non-parametric model, science denier, smoothing, splines, statistical dependence
1 Comment
Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories
I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random walk in a parameter space. … Continue reading
Posted in American Meteorological Association, American Statistical Association, AMETSOC, Arthur Charpentier, atmosphere, diffusion, diffusion processes, dynamic linear models, dynamical systems, environment, geophysics, hurricanes, Kalman filter, Kerry Emanuel, Lévy flights, Lorenz, Markov chain random fields, mathematics, mathematics education, maths, MCMC, mesh models, meteorological models, meteorology, model-free forecasting, Monte Carlo Statistical Methods, numerical analysis, numerical software, oceanography, open data, open source scientific software, physics, random walk processes, random walks, science, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, time series
1 Comment
“Holy crap – an actual book!”
Originally posted on mathbabe:
Yo, everyone! The final version of my book now exists, and I have exactly one copy! Here’s my editor, Amanda Cook, holding it yesterday when we met for beers: Here’s my son holding it: He’s offered…
Posted in American Association for the Advancement of Science, Buckminster Fuller, business, citizen science, citizenship, civilization, complex systems, confirmation bias, data science, data streams, deep recurrent neural networks, denial, economics, education, engineering, ethics, evidence, Internet, investing, life purpose, machine learning, mathematical publishing, mathematics, mathematics education, maths, moral leadership, multivariate statistics, numerical software, numerics, obfuscating data, organizational failures, politics, population biology, prediction, prediction markets, privacy, quantitative biology, quantitative ecology, rationality, reason, reasonableness, rhetoric, risk, Schnabel census, smart data, sociology, statistical dependence, statistics, the right to be and act stupid, the right to know, the value of financial assets, transparency, UU Humanists
Leave a comment
A model of an electrical grid: A vision
Many people seem to view the electrical grid of the future being much like the present one. I think a lot about networks, because of my job. And I especially think a lot about network topologies, although primarily concerning the … Continue reading
Posted in abstraction, American Meteorological Association, anomaly detection, Anthropocene, Bloomberg New Energy Finance, BNEF, Boston, bridge to somewhere, Buckminster Fuller, Canettes Blues Band, clean disruption, climate business, climate economics, complex systems, corporate supply chains, decentralized electric power generation, decentralized energy, demand-side solutions, differential equations, distributed generation, efficiency, EIA, electricity, electricity markets, energy, energy reduction, energy storage, energy utilities, engineering, extended supply chains, green tech, grid defection, Hermann Scheer, Hyper Anthropocene, investment in wind and solar energy, ISO-NE, Kalman filter, kriging, Lawrence Berkeley National Laboratory, leaving fossil fuels in the ground, Lenny Smith, local generation, marginal energy sources, Massachusetts Clean Energy Center, Mathematics and Climate Research Network, mesh models, meteorology, microgrids, networks, New England, New York State, open data, organizational failures, pipelines, planning, prediction markets, public utility commissions, PUCs, rate of return regulation, rationality, reason, reasonableness, regime shifts, regulatory capture, resiliency, risk, Sankey diagram, smart data, solar domination, solar energy, solar power, Spaceship Earth, spatial statistics, state-space models, statistical dependence, statistics, stochastic algorithms, stochastics, stranded assets, supply chains, sustainability, the energy of the people, the green century, the value of financial assets, thermodynamics, time series, Tony Seba, utility company death spiral, wave equations, wind energy, wind power, zero carbon
Leave a comment
On Smart Data
One of the things I find surprising, if not astonishing, is that in the rush to embrace Big Data, a lot of learning and statistical technique has been left apparently discarded along the way. I’m hardly the first to point … Continue reading
Posted in Akaike Information Criterion, Bayes, Bayesian, Bayesian inversion, big data, bigmemory package for R, changepoint detection, data science, data streams, dlm package, dynamic generalized linear models, dynamic linear models, dynamical systems, Generalize Additive Models, generalized linear models, information theoretic statistics, Kalman filter, linear algebra, logistic regression, machine learning, Markov Chain Monte Carlo, mathematics, mathematics education, maths, maximum likelihood, MCMC, Monte Carlo Statistical Methods, multivariate statistics, numerical analysis, numerical software, numerics, quantitative biology, quantitative ecology, rationality, reasonableness, sampling, smart data, state-space models, statistical dependence, statistics, the right to know, time series
Leave a comment
Cory Lesmeister’s treatment of Simson’s Paradox (at “Fear and Loathing in Data Science”)
(Updated 2016-05-08, to provide reference for plateaus of ML functions in vicinity of MLE.) Simpson’s Paradox is one of those phenomena of data which really give Statistics a substance and a role, beyond the roles it inherits from, say, theoretical … Continue reading
Posted in Akaike Information Criterion, approximate Bayesian computation, Bayes, Bayesian, evidence, Frequentist, games of chance, information theoretic statistics, Kalman filter, likelihood-free, mathematics, maths, maximum likelihood, Monte Carlo Statistical Methods, probabilistic programming, rationality, Rauch-Tung-Striebel, Simpson's Paradox, state-space models, statistical dependence, statistics, stochastics
Leave a comment
“Lucky d20” (by Tamino, with my reblogging comments)
Originally posted on Open Mind:
What with talk of killer heat waves, droughts, floods, etc. etc., this blog tends to get pretty serious. When it does, we don’t deal with happy prospects, but with the danger of worldwide catastrophe. But…
Mr Buffett bets the farm
From Dr James Hansen’s blog, of today. So, Mr. Buffett, I am heartened by the words in your last annual report, where you conclude that continued inaction on climate change “is foolhardy.” You wrote: “Call this Noah’s Law: If an … Continue reading
Posted in adaptation, Anthropocene, Bloomberg New Energy Finance, BNEF, climate change, climate disruption, climate justice, corporate litigation on damage from fossil fuel emissions, environment, extended supply chains, games of chance, geophysics, global warming, greenhouse gases, hurricanes, Hyper Anthropocene, insurance, investing, James Hansen, liberal climate deniers, meteorology, oceanography, physics, rate of return regulation, rationality, reasonableness, science, statistical dependence, statistics, sustainability, the right to be and act stupid, the tragedy of our present civilization, the value of financial assets, Wally Broecker, Warren Buffett, zero carbon
Leave a comment
p-values and hypothesis tests: the Bayesian(s) rule
The American Statistical Association of which I am a longtime member issued an important statement today which will hopefully move statistical practice in engineering and especially in the sciences away from the misleading practice of using p-values and hypothesis tests. … Continue reading
Posted in approximate Bayesian computation, arXiv, Bayes, Bayesian, Bayesian inversion, bollocks, Christian Robert, climate, complex systems, data science, Frequentist, information theoretic statistics, likelihood-free, Markov Chain Monte Carlo, MCMC, Monte Carlo Statistical Methods, population biology, rationality, reasonableness, science, scientific publishing, statistical dependence, statistics, stochastics, Student t distribution
Leave a comment
Ah, Hypergeometric!
(“Ah, Hypergeometric!” To be said with the same resignation and acceptance as in “I’ll burn my books–Ah, Mephistopheles!” from Faust.)😉 Dr John Cook, eminent all ’round statistician (with a specialty in biostatistics) and statistical consultant, took up a comment I … Continue reading
