## Eversource withdraws from the Spectra-Algonquin “Access Northeast” pipeline project

(Click on image to see a bigger copy. Use browser Back Button to return to blog.)

Yes!

Now let’s hope the remaining customers for Spectra’s Access Northeast pull out, and FERC denies permission to proceed. Their next meeting is 22nd September 2016.

Update, 2016-08-24

National Grid and the rest of the utilities have pulled out of Spectra-Algonquin’s Access Northeast.

More.

## “Naïve empiricism and what theory suggests about errors in observed global warming”

A post from one of my favorite statistics-oriented bloggers, Variable Variability, dealing with a subject too casually passed over.

## “Understanding Climate Change with Bill Nye”, on Dr Neil deGrasse Tyson’s “Star Talk”

Bill Nye hosts Dr Neil deGrasse Tyson‘s Star Talk Radio, featuring climate change and NASA’s Dr Gavin Schmidt. (See also RealClimate.)

## ECS2x, land, sea, and all that

from http://dx.doi.org/10.1126/science.1203513

P.S. I wrote more here. Reproduced below …

Practical likelihood functions are very flat-topped, so the idea that a maximum likelihood function (MLE) can be confined to a point is a theoretical mirage. See Chapter 3 of S. Konishi, G. Kitagawa, Information Criteria and Statistical Modeling, Springer, 2008. Even if you want to set aside Bayesian considerations, whose priors tend to sharpen the posteriors, the best you can do is expected likelihoods, because likelihoods in practice, just like p-values, are random variables. Accordingly, the MLE is a neighborhood, because a point has probability mass zero.

Besides, … the question of multimodality [wasn’t addressed]. Actual Expected Climate Sensitivity is a combination of the densities over oceans and land, each of which have different distributions and modes. (See https://goo.gl/pB7H24 which is from http://dx.doi.org/10.1126/science.1203513) Accordingly, their combination is (at least) bimodal. Ocean ECS has 4 modes. Land ECS has 2 modes, one slightly higher than the other, the higher being at +3.4°C and the second at about +3°C. Worse, the variance of land ECS is over twice than of oceans.

Finally, what you should be looking at is the ECS2x over land, not combined. Even if granted to want to go with the location of the highest mode, that’s +3.4°C.

## A litany of climate depression

Bill Nye talks about the show, here.

## “Disrupt climate disruption”

From Science Music Videos

And if you have the time, a 52 minute movie …

Power concedes nothing without a demand.

“No leader is coming to save us [from climate disruption].”

## Carbon Sinks in Crisis — It Looks Like the World’s Largest Rainforest is Starting to Bleed Greenhouse Gasses

This is the kind of thing that’s expected of a +3C world, although the idea of it being a threshold phenomenon is a bit unrealistic. So, it’s expected that these sinks might, overall, start releasing their sequestered Carbon, one here one year, another there another year. But if the biggest sinks start releasing theirs first, well, this is one of the Climate Surprises the IPCC and the U.S. National Climate Change assessment talk about. And they are not at all good.

Back in 2005, and again in 2010, the vast Amazon rainforest, which has been aptly described as the world’s lungs, briefly lost its ability to take in atmospheric carbon dioxide. Its drought-stressed trees were not growing and respiring enough to, on balance, draw carbon out of the air. Fires roared through the forest, transforming trees into kindling and releasing the carbon stored in their wood back into the air.

These episodes were the first times that the Amazon was documented to have lost its ability to take in atmospheric carbon on a net basis. The rainforest had become what’s called carbon-neutral. In other words, it released as much carbon as it took in. Scientists saw this as kind of a big deal.

This summer, a similar switch-off appears to be happening again in the Amazon. A severe drought is again stressing trees even as it is fanning…

View original post 1,144 more words

## An Energy Revolution

Professor Mara Prentiss speaks at Harvard on the possibility of an “energy revolution”:

Update, 2016-08-16

Although I am not a PhD professor like Professor Prentiss, nor am I associated with an institution as esteemed as Harvard University, I disagree with her point regarding the need for natural gas, based upon my studies of the solar energy business during the last two years. I also assuredly hope that is not the only way to transition to zero Carbon energy, because from what I know of the climate science, we could be in very, very serious trouble if we need to go through an intermediate CO2-spewing step which will persist for another few decades.

## Can the City of Boston adapt to and help mitigate climate disruption?

(On “Less Science and More Social Science” at And Then There’s Physics)

And Then There’s Physics is one of my favorite blogs discussing climate disruption and related policy (in my climate blogroll, for instance). There was a recent post regarding another post by a science blogger called Stoat (one William Connolley) on the limitations of science for dictating mitigation and adaptation policy. Read there for context.

But along the way, a Commentor, mt at 13th August 2016 at 6:25 p.m. where mt, cited the 1979 `Charney Commission report’, suggests science can and has done little more, even with the IPCC. I composed a Comment which suggested at least one city, Boston, was trying to enlist science in its detailed response and planning.

That Comment apparently did not make it through moderation at ATTP, or got lost through a technical glitch, as sometimes happens. I worked on it a bit, so am reproducing it here instead. (As can be seen by the Comment below, apparently there was a technical glitch, and that Comment has now been posted.)

@mt,

Well, the City of Boston is engaged in a pretty deliberate process to ascertain climate impacts, what should City policy and planning dictate, especially with respect to sea level rise and storm surge, and needed investments. It is informed by science, and climate projections for Boston, followed by three additional reports, an Integrated Vulnerability Assessment, a detailing of Resilience Strategies, and a Final Report and Implementation Roadmap. The last three appear to be late, but there is a hard stop of sorts in the form of a Climate Vulnerabilities & Solutions Symposium on the 15th of September, which I am attending.

Attendees will include representatives from local financial firms, banks, insurers and re-insurers, as well as businesses, utilities, real estate people, government people, NGOs and attorneys. There already was a presentation of the Climate Projections Consensus at which there were many representatives of these stakeholders.

Come September, it will be interesting to see how these groups think about the problem, and where they are landing in terms of a mix of the three basic choices,

1. wait-and-see, with willingness to take on and deal with damage as it comes,
2. makes some preparations, but basically remain-in-place, or
3. preparare to abandon the present location of the City, and begin preparations to assess where to go.

This is in part because:

I like Bank of England head Mark Carney’s description that “Climate change is an economic problem.” He wants to avoid a Minsky moment.

The economy is a wholly owned subsidiary of the environment, not the other way around.

Gaylord Nelson

It should be noted as well that the City of Boston has volunteered itself to host a United States-China Climate Summit in 2017. Whether that applies additional pressure, as I suspect, or gives the City cover for delay and greenwashing is anyone’s guess. We’ll see in September.

## Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories

I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random walk in a parameter space. The near-term goal is to understand how to apply these techniques to series of categorical data, and understand the relationship between, say,
``` W. Li, ``Markov chain random fields for estimation of categorical variables'', Mathematical Geology (2007) 39: 321–335 DOI 10.1007/s11004-007-9081-0. ```
and
``` K. V. Mardia, C. Goodall, E. J. Redfern, F. J. Alonso, ``The kriged Kalman filter'', Test, December 1998, 7(2), 217–282 DOI 10.1007/BF02565111. ```
Along the way, I came across a mention of work done by Christophe Denuse-Baillon in his actuarial thesis which appears to study and describe a method of simulating North Atlantic hurricane tracks using a Markov spatial process calibrated with historical tracks. The idea, essentially, is to make a random walk from a suitably chosen starting point, taking steps conditional upon relative frequencies of directions taken from the current step. In other words, the Markovian assumption of conditional independence is made so the next step depends only upon the probabilities of taking a step in directions centered on the current one.

I would not know the details, since Denuse-Baillon’s thesis is in French, but Professor Arthur Charpentier introduced the idea and provided some R code for implementing Denuse-Baillon’s idea on actual hurricane tracks. Now the thing is the datasets he drew upon have naming mistakes in them which need to be repaired in order to use them, and his blog post did not identify these, although he did note they existed. But, more seriously, he did not provide a complete, runnable set of R code to reproduce his results, something which would be quite useful to students of the technique and the problem.

This blog posting remedies that providing runnable R code for replicating his results (a tarball), and fixing the mistakes in the datasets on the fly. Note that the R relies upon the XML, maps, ks, and RColorBrewer packages which also need to be installed.

The general technique is interesting in that it offers a non-parametric, indeed, model free way of forecasting hurricane tracks.

I have reproduced the figures Professor Charpentier showed below:

Larger versions of these figures can be seen by clicking on any one of them, and then using your browser Back Button to return to the blog.

The R code has been cleaned up some, and made a bit more robust.

Robert Grant also remarked upon Professor Charpentier’s work.

Note that it is still possible that when trajectories are generated, if a poor starting point is chosen, the generation step can take a while.

I also want to make note of the spMC package by Luca Sartore, available via CRAN, which he describes here. Abstract:

Currently, a part of the R statistical software is developed in order to deal with spatial models. More specifically, some available packages allow the user to analyse categorical spatial random patterns. However, only the spMC package considers a viewpoint based on transition probabilities between locations. Through the use of this package it is possible to analyse the spatial variability of data, make inference, predict and simulate the categorical classes in unobserved sites. An example is presented by analysing the well-known Swiss Jura data set.

## Living deliberately in Washington, D.C. (courtesy of The Atlantic magazine)

The adventures of Keya Chatterjee and her family living free of Pepco. Courtesy of The Atlantic magazine.

## Energy Democracy

I’ve actually written about this before, but John Farrell of the ILSR (“Institute for Local Self-Reliance” a famous Emerson essay, by the way) presents an up-to-date synthesis of developments, incorporating policy as well as Tony Seba-like, Hermann Scheer-like, and Michael Osborne-like insights.

By the way, the dreaded duck curve which utilities wonks and even some IEEE engineers from the PES I’ve listened to (more details here) does not seem to be materializing in two of the strongest renewables markets in the United States.

Update, 2016-08-21

From CleanTechnica:

Distributed Power

It’s a matter of political and philosophical debate, but I agree with the idea that society is generally better off when socioeconomic and political power are distributed. (Granted, a benevolent dictator can be a wonderful gift for a society, but most dictatorships don’t tend to be very benevolent from what I’ve seen.)

While we do live in a somewhat democratic society (in the US, Canada, UK, Europe, Australia, India, Korea, Japan, or wherever you are probably reading from), there’s no doubt that money = power, and people with more money or representing more money have more power in politics and society.

With regard to this matter, we often think of powerful people and companies in the telecommunications, media, banking, and real estate industries. Clearly, though, these aren’t the only ones trying to steer more cash to their executives than to society as a whole.

Of course, with many utilities being regulated monopolies, these are powerful giants as well (no pun intended). Despite the fact that they are regulated, the vast majority of us can’t name the people who regulate them, and there is rampant corruption in the sector. Largely, we don’t even know what they’re doing. I think we typically take utilities for granted and leave their work almost invisible — they’re there, we have to pay them to keep the lights and computers on, someone is watching over them to make sure they don’t fleece us (too much), etc.

A more obvious “enemy of the societal good” is the fossil energy industry. Burning coal and natural gas kills millions of people prematurely every year. We somehow accept burning these fossils as a necessity of modern life (though, given the state of clean technologies like solar and EVs, we no longer should), but we also know that these industries work hard to not clean up their processes and emit less pollution. They lobby government and fight huge wars against regulation with millions and millions of dollars that could have just gone toward protecting more lives from pollution. But hey, what can we do?

To find out, read their article.

Update: 2016-08-23

From ILSR, again:

I wouldn’t presume to define energy democracy for all those using the term, but I think those of us that use it share these common principles:

• Energy democracy means both the sources (e.g. solar panels) and ownership of energy generation are distributed widely.
• Energy democracy means that the management of the energy system be governed by democratic principles (e.g. by a public, transparent, accountable authority) that allows ordinary citizens to have a say. This means that communities that wish greater control over their energy system (via municipalization of utilities, for example) should have minimal barriers to doing so.
• Energy democracy means that the wide distribution of power generation and ownership, and access to governance of the energy system be equitable by race and socioeconomic status.

## “Holy crap – an actual book!”

You’ll find links to Cathy O’Neil’s important book in the Blogroll here, as well as a link to reviews of it.

I have not read it yet. While I have pre-ordered it, it’s not available. I have read the reviews, all favorable. I suspect it expresses a lot of concern that statisticians, including myself, have long expressed regarding “data mining” (data dredging), and inferences derived from “big data”, including a lack of concern about the quality of the data, forgetting about cleaning, and whether or not, despite the size of a dataset, there is enough data to warrant a strong estimate or inference in a particular case.

These arguments have been made by many, like Simply Statistics‘ “Why big data is in trouble: they forgot about applied statistics” from 2014. Or Bajorski’s “Applied Statistics Comes to the Rescue of Big Data”. And the wave of emphasis upon pure machine learning methods of late does not help: These obfuscate and confound, making it harder to understand what’s really going on.

Anyway, I very much look forward to reading the book!

Yo, everyone! The final version of my book now exists, and I have exactly one copy! Here’s my editor, Amanda Cook, holding it yesterday when we met for beers:

Here’s my son holding it:

He’s offered to become a meme in support of book sales.

Here’s the back of the book, with blurbs from really exceptional people:

In other exciting book news, there’s a review by Richard Beales from Reuter’s BreakingViews, and it made a list of new releases in Scientific American as well.

Endnote:

I want to apologize in advance for all the book news I’m going to be blogging, tweeting, and otherwise blabbing about. To be clear, I’ve been told it’s my job for the next few months to be a PR person for my book, so I guess that’s what I’m up to. If you come here for ideas and are turned off by cheerleading, feel…

View original post 34 more words

## Dramatis personæ: How to do zero Carbon emissions at a residence (Westwood, MA)

(To see a larger picture for this and all images, click on image and then use browser Back Button to return to blog.)

Our external meter is now reading negative.

Peak was 2850 in April 2016. Solar panels were grid connected on 31st December 2015.

Now, and for the last couple of months, our electric bill has been negative:

This is the trend heading to zero on the meter, including a calculation of household electricity consumption. We heat with electricity, heat hot water with electricity, and have an induction stove:

How?

Air source heat pumps for heating and cooling:

You can find the latest on these systems here. We have models ASU12RLF, AUU12RLF, ASU9RLF.

Air source hot water heater (General Electric):

And, of course, 29 SunPower PV panels installed by RevoluSun:

So far we’re generated 6.6 megawatt-hours:

Our inverter:

And this is the oil heating system we orphaned to do this:

## TOO LATE: “There will be no golden age of [natural] gas”

Last month, Tom Randall at Bloomberg New Energy Finance (“BNEF”) profiled a new forecast which shows costs for zero Carbon energy and energy are plummetting so fast that coal, oil, and natural gas will begin their terminal decline within a decade.

Since 2008, the single most important force in U.S. power markets has been the abundance of cheap natural gas brought about by fracking. Cheap gas has ravaged the U.S. coal industry and inspired talk of a “bridge fuel” that moves the world from coal to renewable energy. It doesn’t look like that’s going to happen.

The costs of wind and solar power are falling too quickly for gas ever to dominate on a global scale, according to BNEF. The analysts reduced their long-term forecasts for coal and natural gas prices by a third for this year’s report, but even rock-bottom prices won’t be enough to derail a rapid global transition toward renewable energy.

“You can’t fight the future,” said Seb Henbest, the report’s lead author. “The economics are increasingly locked in.” The peak year for coal, gas, and oil: 2025.

In addition to the pace of technology improvements in solar, storage, and wind, US\$8 trillion is estimated to be poised to be dropped into renewables energy, even if, through 2040, people will, dare I say, waste US\$2 trillion developing more fossil fuels. By 2027, BNEF estimates that building new wind and solar power will be cheaper than even operating fossil fuel plants. And capacity factors for peaking and standard generation natural gas plants will be hit by the increasing availability of extremely low cost zero Carbon energy, meaning the capital costs of the fossil fuel plants will skyrocket. These assets will rapidly be stranded. And this is just on price: There’s no accounting for what the impact will be of Carbon fees, which are looking increasingly plausible, if only to raise monies to clean up the mess the fossil fuel industry has made of our atmosphere.

And, from what I have seen of fossil fuel companies, their infrastructure builders, and their PR people, it couldn’t happen to a nicer bunch of folks. They deserve it. No tears here. A new kind of climate justice.

But, as far as climate disruption goes, people should not get complacent. Even if all this happens on this schedule, it won’t happen fast enough to prevent serious disruption of life and business due to climate effects. Accordingly, we need to push. And if the public are convinced to waste their share of that US\$2 trillion on fossil fuels and their infrastructure, this could seriously slow us down making the change we need to make.

Taking the CO2 out of atmosphere once it’s there is not easy and it is very expensive.

Important Update

The Massachusetts Supreme Judicial Court today roundly denied the Republican Governor Baker, his Department of Public Utilities, Eversource, National Grid, and pipeline companies, notably Spectra Energy, the vehicle whereby ratepayers would be charged to pay for construction of additional natural gas capacity to Massachusetts for the generation of electricity.

Perhaps most importantly, however, the department’s order would reexpose ratepayers to the very types of risks that the Legislature sought to protect them from when it enacted the restructuring act. Both the DOER and the department noted that gas-fired generating businesses are unwilling to assume the risks associated with long-term gas pipeline capacity contracts because there “is no means by which they can” assure recovery of those contract costs. Shifting that risk onto the electric ratepayers of the Commonwealth, however, is entirely contrary to the risk-allocation design of the restructuring act. Equally unavailing is the department’s finding that the order does not contravene the policy embodied in the restructuring act because it does not allow the use of ratepayer funds to construct a power plant. D.P.U. 15-37, at 27. As prior decisions by this court and the department make clear, power plant construction is only one aspect of the electric generation market, and in enacting the restructuring act, the Legislature sought to separate all aspects of generation from all aspects of distribution. See, e.g., D.T.E. 98-13, at 4; D.T.E. 98-84, at 1. Moreover, the department itself has recognized that fuel procurement and planning is an integral component of the generation business, as evidenced by its exemption of electric distribution companies from § 69I. Indeed, by some estimations, fuel-related costs constitute seventy-five per cent of a natural gas-fired plant’s generation costs. 3 World Scientific Handbook of Energy 72 (G.M. Crawley ed., 2013). Accordingly, prior to the enactment of the restructuring act, the department required electric companies to consider both the type and amount of fuel they would use to generate power when they calculated whether they could supply enough electricity to match expected demand. We agree with the plaintiffs that if the restructuring act does not allow electric distribution companies to finance investments in electric generation, it cannot be reasonably interpreted to permit those companies to invest in infrastructure unrelated to electric distribution service. Accordingly, we reject the department’s reasoning. See Cardin v. Royal Ins. Co. of Am., 394 Mass. 450, 456-557 (1985) (agency’s interpretation of statute “hardly persuasive where [it] violates the language and policy of the statute,” [quotation and citation omitted]).

The department’s interpretation of the statute as permitting electric distribution companies to shift the entire risk of the investment to the ratepayers is unreasonable, as it is precisely this type of shift that the Legislature sought to preclude through the restructuring act. Contrast D.P.U. 12-77, at 28 (Mar. 15, 2013) (“The legislation restructured the electric industry in the state by providing incentives to investor-owned electric distribution companies to divest their generating assets and by adopting a competitive market structure for the generation and purchase of electricity. This restructuring shifted the risks of generation development from consumers to generators, who are better positioned to manage those risks”).

Our interpretation of the restructuring act is supported by the Legislature’s own actions since the law’s enactment. That is, where the Legislature has sought to override the risk allocation policy of the act, it has done so expressly. First, in 2008, through enactment of the Green Communities Act, St. 2008, c. 169, the Legislature directed electric distribution companies to seek proposals from renewable energy developers, and, if they received reasonable proposals, to enter into ratepayer-backed long-term contracts to buy the renewable power. See St. 2008, c. 169, § 83. The Legislature concluded that such contracts were necessary to “facilitate the financing of renewable energy generation facilities.” Alliance to Protect Nantucket Sound, Inc. v. Department of Pub. Utils. (No. 1), 461 Mass. 166, 168 (2011). Importantly, in enacting the Green Communities Act, the Legislature explicitly provided the department with the authority to review and approve the ratepayer-backed renewable energy contracts. St. 2008, c. 169, § 83 (“[a]ll proposed contracts shall be subject to the review and approval of the department of public utilities”).

The Green Communities Act represents a legislatively created exception to the restructuring act’s general prohibition on electric distribution companies owning generation assets. To facilitate promotion of renewable energy in the Commonwealth, the Legislature allowed each distribution company to construct, own, and operate twenty-five megawatts of solar energy before January 1, 2009, and 50 megawatts after January 1, 2010. St. 2008, c. 169, § 58. Section 58 further provided that an electric distribution company had to obtain prior approval for cost recovery from the department in order to recover construction costs of a solar generation facility. Id. Although the statute has since been amended, it continues to provide an express, limited exemption from the restructuring act. See St. 2012, c. 209, § 17.
.
.
.
Here, the department’s stated motive in issuing the order is to correct a perceived failure of market-based incentives to encourage wholesale generators to contract for adequate pipeline capacity. However, its means of doing so, namely by reallocating risk onto the ratepayers, is clearly prohibited by legislative policy. Thus, no matter how salutary the department may claim its policy aims to be, its order contravenes the fundamental policy embodied in the restructuring act and cannot stand. See Utility Air Regulatory Group v. Environmental Protection Agency, 134 S. Ct. 2427, 2446 (2014) (agency authority to interpret ambiguities in enabling statute “does not include a power to rewrite clear statutory terms to suit its own sense of how the statute should operate”); Wakefield Teachers Ass’n v. School Comm. of Wakefield, 431 Mass. 792, 802 (2000) (fundamental policy decisions are province of Legislature, and not coordinate branches of government).

## A model of an electrical grid: A vision

Many people seem to view the electrical grid of the future being much like the present one. I think a lot about networks, because of my job. And I especially think a lot about network topologies, although primarily concerning the Internet.

Presently, most utilities’ electrical grids are hub-and-spoke topologies, like Boston’s somewhat wanting subway system, where there is a center, and distribution is fed from it, and generation feeds it from an array of generators who are paid for the privilege and bid to do so. It is overseen by ISOs like ISO-NE. Sometimes there isn’t enough generation and one ISO needs to borrow from the next, but that’s rare and, in this market-based system, expensive. Bidding is done day-ahead.

What I envision is a completely different topology.

The equilibrium state of such a grid or network would consist of spatially separated clusters, roughly proportional to population, with some cleaving along political boundaries. Each cluster would have its own generation sources — wind, solar, storage — and, of course, its own demand. Roughly half the time (or more!) each cluster gets all the power it needs from its own generation. The rest of the time, the cluster draws power from available power in nearby clusters, coordinated as a super-cluster. The great majority of the time, say 90+%, each super-cluster is completely self-sufficient. There may be community wind or solar generation at the super-cluster level which ties. It is entirely possible that the distribution lines and substations at the super-cluster are owned by an entity not associated with what we would identify as a “utility”. At best (or worst) it would be like a municipal utility or energy cooperative. (These already exist in Massachusetts.) They maintain the distribution network. While they are a kind of monopoly, like utilities, they have a smaller population to serve and, accordingly, need to answer for the level of service they do or not provide. Their board might even be elected.

When a super-cluster fails to provide for its own energy, it can either request it from neighboring super-clusters, or it can approach what is the vestige of the present utility, the vestige of a hub or grid. Such an entity would be economically much smaller than today, having sold off most of its assets to super-clusters, but its role would remain, albeit much smaller than that of today. Its political influence would also be accordingly smaller.

There is also a much reduced need for an ISO in this model. Rather than a centrally planned and centrally controlled manager of energy, the clusters system can be mathematically designed and implemented via control theory to be self-balanced and self-regulating. Accordingly, there is little need for command decisions, although there continues to be a need for monitoring and reporting to national authorities and agencies.

Naturally, transitioning from the present central system to such a clustered approach will have its bumps and pitfalls. I think this will be technology and price driven, and there will be entrepreneurs who will create the local storage and digital energy overlay networks to make this happen. It would probably go easier if there were an oversight agency or agencies. New York State appears to be trying to do something like that, but I’m skeptical that Massachusetts, for instance, can, since its governance is tied up and torn between a number of fierce vested interests.

I bet New York will get there first, and I bet New York’s economy will be much the better for it, at least than that of Massachusetts.

Finally, such a system, hierarchically decomposed, will be far more robust and resilient than the present one, if much less profitable, because it depends less upon extended supply chains. Its price volatility will be a fraction of the volatility in the present system, offering less reward for arbitrage. It will also be vastly more efficient in its use of energy. There will be no need for natural gas pipelines. These are good.

This is the kind of “grid defection” or, rather, grid transformation most policy leaders don’t imagine.

(Below, some gratuitously included clips from the Canettes Blues Band, literally, a group of LHC physicists who play the blues.)

## Bayesian blocks via PELT in R

The Bayesian blocks algorithm of Scargle, Jackson, Norris, and Chiang has an enthusiastic user community in astrostatistics, in data mining, and among some in machine learning. It is a dynamic programming algorithm (see VanderPlas referenced below) and, so, exhibits optimality when used without performance-related shortcuts. There is an implementation by Scargle and others in MATLAB, and a popular version exists in the astropy and astroML modules for Python. Scargle introduced the idea about 2001, and has repeatedly improved it. I searched for a comparable implementation for R, the programming language I use most heavily and could not find one.

In time, however, I discovered the work of Killick, Fearnhead, and Eckley, led to it by an oceanographer colleague. They build upon the work of Jackson, Scargle, Barnes, Arabhi, Alt, Gioumousis, Gwin, Sangtrakulcharoen, Tan, and Tsai, but find a pruning step which lets the algorithm achieve the same result but in linear time. Their algorithm, known as PELT, has been incorporated into two R packages on CRAN, changepoint and changepoint.np, and has found use in oceanography and other areas. Still, a one-for-one substitute for the Scargle-Jackson-Norris-Chiang Bayesian blocks was missing, and it is the purpose of this post to remedy that.

• R. Killick, P. Fearnhead, And I. A. Eckley, “Optimal detection of changepoints with a linear computational cost”, Journal of the American Statistical Association, December 2012, 107(500), DOI: 10.1080/01621459.2012.737745
• R. Killick, I. A. Eckley, K. Ewans, P. Johnson, “Detection of changes in the characteristics of oceanographic time-series using changepoint
analysis”, Ocean Engineering, 2010, 37(13), 10.1016/j.oceaneng.2010.04.009
• R. Killick, I. A. Eckley, “changepoint: An R package for changepoint analysis”, Journal of Statistical Software, June 2014, 58(3), 10.18637/jss.v058.i03
• B. Jackson, J. D. Scargle, D. Barnes, S. Arabhi, A. Alt, P. Gioumousis, E. Gwin, P. Sangtrakulcharoen, L. Tan, T. T. Tsai, “An algorithm for optimal partitioning of data on an interval”, IEEE Signal Processing Letters, February 2005, 12(2), 10.1109/LSP.2001.838216
• J. D. Scargle, J. P. Norris, B. Jackson, J. Chiang, “Studies in astronomical time series analysis. VI. Bayesian Block representations”, The Astrophysical Journal, 2013 February 20, 764(167), 10.1088/0004-637X/764/2/167
• J. VanderPlas, “Dynamic programming in Python: Bayesian Blocks”, Pythonic Perambulations, 12 September 2012, a blog entry
• bayesian_blocks, a component of astropy.stats, the astropy module for Python
• 4.1 Bayesian Blocks: Histograms the right way, a component of 4. Unsupervised Learning: Density Estimation, the astroML module for Python
• J. D. Scargle, “Bayesian Blocks in two or more dimensions: Image segmentation and cluster analysis”, arXiv:math/0111128v1

Jake VanderPlas is one of the astroML authors and, specifically that of the Bayesian blocks implementation therein. He offers a description of the Scargle algorithm, explaining it as an instance of a dynamic programming code pattern. He also offers a clever mixture distribution consisting of five Cauchy distributions for illustrating his code, a mixture distribution I have adopted for illustrating this, my implementation of Bayesian blocks in R. (R code is at link.)

Comparing its performance to that of a standard histogramming facility, truehist in the MASS package, we see the following:

(Click on figure to see a larger image, then use browser Back Button to return to blog.)

The result from VanderPlas’ implementation can be seen at his blog, just above the Conclusion at its bottom.

But VanderPlas’ mixture also can serve as a series for another test. Bayesian blocks has been proposed as a signals representation as well, one which can be used for compression as well as specifying features. The implementation here, based upon the Killick-Fearnhead-Eckley PELT, can do that as well, as can be seen by the figure below:

(Click on figure to see a larger image, then use browser Back Button to return to blog.)

The code is available from my Google space as a gzip‘d R file. Please feel free to use for anything you like, and, not only would I be happy to answer questions about it, I’d like to know what kinds of things you are using it for, or receiving suggestions for improvements. Please comment below!

I am also intending to generalize the code so it can be used for multidimensional Bayesian blocks, as Scargle has suggested (in J. D. Scargle, “Bayesian Blocks in two or more dimensions: Image segmentation and cluster analysis”, arXiv:math/0111128v1), but, of course, based upon the Killick-Fearnhead-Eckley PELT. My particular interest is cluster-finding.

## “Full-depth Ocean Heat Content” reblog

This is a re-blog of an excellent post at And Then There’s Physics, titled Full-depth OHC or, expanded, “full-depth ocean heat content”.

Since my holiday is now over, I thought I might briefly comment on a recent paper by Cheng et al., called Observed and simulated full-depth ocean heat-content changes for 1970–2005. John Abraham, o…

Source: Full-depth OHC

## Richard Somerville, UCSD, Scripps: “The science is becoming more widely accepted”

By Richard Somerville, emiritus professor of Oceanography from Scripps Institution of Oceanography. See the site he helps build and run regarding communication regarding change.

## “Stochastic Parameterization: Towards a new view of weather and climate models”

Judith Berner, Ulrich Achatz, Lauriane Batté, Lisa Bengtsson, Alvaro De La Cámara, Hannah M. Christensen, Matteo Colangeli, Danielle R. B. Coleman, Daan Crommelin, Stamen I. Dolaptchiev, Christian L.E. Franzke, Petra Friederichs, Peter Imkeller, Heikki Järvinen, Stephan Juricke, Vassili Kitsios, François Lott, Valerio Lucarini, Salil Mahajan, Timothy N. Palmer, Cécile Penland, Mirjana Sakradzija, Jin-Song Von Storch, Antje Weisheimer, Michael Weniger, Paul D. Williams, Jun-Ichi Yano, Stochastic Parameterization: Towards a new view of weather and climate models, Bulletin of the American Meteorological Society, published online $19^{th}$ July 2016,

Abstract

Stochastic parameterizations — empirically derived, or based on rigorous mathematical and statistical concepts — have great potential to increase the predictive capability of next generation weather and climate models.

The last decade has seen the success of stochastic parameterizations in short-term, medium-range and seasonal forecasts: operational weather centers now routinely use stochastic parameterization schemes to better represent model inadequacy and improve the quantification of forecast uncertainty. Developed initially for numerical weather prediction, the inclusion of stochastic parameterizations not only provides better estimates of uncertainty, but it is also extremely promising for reducing longstanding climate biases and relevant for determining the climate response to external forcing.

This article highlights recent developments from different research groups which show that the stochastic representation of unresolved processes in the atmosphere, oceans, land surface and cryosphere of comprehensive weather and climate models (a) gives rise to more reliable probabilistic forecasts of weather and climate and (b) reduces systematic model bias.

We make a case that the use of mathematically stringent methods for the derivation of stochastic dynamic equations will lead to substantial improvements in our ability to accurately simulate weather and climate at all scales. Recent work in mathematics, statistical mechanics and turbulence is reviewed, its relevance for the climate problem demonstrated, and future research directions outlined.

And five related papers, from another field:

## Natural gas: The Zaphod Beeblebrox of energy

Amber Lin at The Bulletin of the Atomic Scientists describes the two-headed character of natural gas plants needed to implement “natural gas as a bridge fuel”, and sketches the stark reality proponents of that argument are embracing if they are serious about using natural gas, whether for electricity or heating, to reduce greenhouse gas emissions.

The basic fact is that in order to serve as a proper “bridge”, natural gas infrastructure would need to be decommissioned by 2050, including ceasing flows of the gas through the elaborate pipelines which criss-cross the United States. That’s because emission limits for CO2 dictated by Nature cannot be met otherwise, with 450 ppm CO2, just 40 ppm higher than where we are now, corresponding to the widely accepted +2°C warming limit. And, as that is unlikely, if we want to limit warming to +3°C 650 ppm is the overall limit, and +3°C brings us into a highly uncertain, dangerous, and eventually ice-free world. In particular, we might lose control of a portion of the warming process, since large natural stores of CO2 are quite likely to be breached and begin leaking at those temperatures.

The Presidential commission on the matter also sketched the key problem with using a “bridge fuel” mechanism to reach targets like this, namely, “A slow start leads to a crash finish”, meaning that to hit these targets, the abandonment of fossil fuels and their infrastructure must be pursued much more quickly than if we start early. I daresay, none of the proposals for new natural gas generation have incorporated operating lifetimes which abruptly end in 2050, or depreciation schedules which reflect that. In fact, the new Massachusetts Salem Harbor gas-powered electricity generator has a lifetime up through 2080.

Amber Lin tells how there are really two incompatible kinds of natural gas plants for electricity generation:

When constructing a new natural gas power plant, there are two options: a combined cycle or an open cycle. A combined-cycle power plant produces electricity with relatively high efficiency and low carbon emissions: When the gas burns, it heats and compresses air to spin a turbine and power a generator. A heat recovery system captures waste heat, which is routed to a nearby steam turbine to generate even more power. Combined-cycle plants have low operating costs, but because high capital costs must be offset, these plants are built to produce baseload power—available 24 hours a day. Open-cycle gas turbine plants lack the steam cycle, so their thermal efficiency is much lower, and their carbon emissions per unit of electricity generated are slightly higher. Their running costs are much higher than a combined-cycle plant, but they have a much lower start-up cost, so they are often built as “peakers,” plants that run only to support other power infrastructure during hours of high demand or when solar or wind isn’t available.

Considering the two choices in the larger context of natural gas as a “transition fuel,” a dilemma appears: To build the bridge, combined-cycle is what is needed—a consistent, efficient, power source that can effectively replace coal. But for a combined-cycle natural gas plant to be economically feasible, it would typically need 15 to 20 years to make up for start-up costs, and even longer to become profitable. This means that a combined-cycle plant built in 2016 would break even no sooner than 2031, and would have to run for several more decades to be a worthwhile investment. Levi’s 2030 limit for peak emissions, and roughly 2050 limit for zero emissions, translate to major fossil fuel reductions after 2030. Owners and backers, however, will not want to shut down gas plants that are just beginning to generate a profit. Thus, building combined-cycle plants in 2016 without an explicit understanding of their necessarily temporary nature—and with no financial incentives for early closures in the future—defeats the purpose of natural gas as a “transition fuel.”

Why not focus on open-cycle plants instead? While “peakers” make sense as backups for future renewable energy sources, they don’t make sense right now. In the current infrastructure, they can only run for a couple hundred hours a year before they cost more than they can earn; this is not nearly enough to displace coal. Closed-cycle plants can help build the bridge but cannot close it, and open-cycle plants can help close the bridge but cannot build it. Neither type of plant is both economically feasible in the long run, and powerful enough to meet today’s demand while cutting emissions in time to mitigate climate change. However, when natural gas is branded as a “transition fuel” in politics and in popular media, this crucial detail is rarely mentioned.

So natural gas plants are the Zaphod Beeblebrox of electricity generation, as they are duplicitous and their purpose is to distract from the true goals of natural gas infrastructure expansion, to prolong the day when fossil fuel assets are stranded because of government action to mitigate climate change, or, as increasingly plausible, it is taxed for its Carbon.

Luckily Arthur’s Betelgeusean friend, Ford Prefect, a roving researcher for that illustrious interstellar travel almanac The Hitchhikers Guide to the Galaxy, was more of an optimist. Ford saw silver linings where Arthur saw only clouds and so between them they made one prudent space traveller, unless their travels led them to the planet Junipella where the clouds actually did have silver linings. Arthur would have doubtless steered the ship straight into the nearest cloud of gloom and Ford would have almost certainly attempted to steal the silver, which would have resulted in the catastrophic combustion of the natural gas inside the lining. The explo­sion would have been pretty, but as a heroic ending it would lack a certain something, i.e. a hero in one piece.

(An extract from The Hitchhikers Guide to the Galaxy.)

## The Presidential betting markets

Someone blatantly misrepresented the U.S. Presidential election betting markets in a Google+ comment thread tonight, and I wanted to bring these forward, here.

Done.

No doubt some supporters of Trump will argue “God is on our side, and so these heathen markets cannot be correct”. I’ll bet.

Update, 2016-07-24

Current odds on Betfair.

## Now, if we could only say the same thing about Massachusetts …

Massachusetts is supposed to be a Blue State.

Massachusetts is supposed to be concerned about the environment, full of tree-hugging eco-weenies (like myself!), and sprouting solar panels from every other rooftop.

Massachusetts is supposed to have aggressive support for zero Carbon energy, including incentives, SRECs, and so on.

But facts are different.

59% of Massachusetts electricity comes from explosive methane (“natural gas” to those of you who prefer industry adverts). This is a potent greenhouse gas which, in 20 year timeframes, is 90x worse than CO2 for climate disrupting radiative forcing. (See https://667-per-cm.net/about if you have doubts.) Natural gas ain’t granola. And the calculations which suggest it is better for the environment are, in my opinion, whacked and bupkis. Set aside upstream impacts from fracking. Not all methane is burnt when it goes up your chimney for heating, nor in generating plants. There are big leaks throughout the Boston metropolitan area which the utilities will fix “if they are dangerous”, but they don’t consider greenhouse gas emissions dangerous. And we all known we have to transition off of fossil fuels, for the good of ourselves, a coastal state as we are, and the moral good of the planet and people on it, not to mention the recently affirmed requirements of the Global Warming Solutions Act (“GWSA”). The Union of Concerned Scientists says we are getting overdependent upon natural gas. And the comparison with coal as a benefit is a logical fallacy of “the worst negates the bad”.

That’s quite different than Texas. Yes, Texas. Home of cowboy boots, and guns, and Spectra Energy.

1. Texas holds the record for all-time wind energy production.
2. The benefits of wind are estimated at \$3.3 billion annually.
3. Texas was the first US state to reach 10,000 megawatts of wind power generating capacity.
4. Texas was One of the First U.S. States to Require a Certain Amount of Electricity Come from Renewable Energy Sources. (As was Massachusetts, to its credit.)
5. Texas wind power is cheaper than fossil fuels.
6. The Texas wind industry employs more than 24,000 workers.

Without going really big on offshore wind and solar, Massachusetts could be being just being a bunch of chumps. And I often wonder if Spectra Energy isn’t trying to dump their explosive methane here, because people at home know better. Or it could be Massachusetts citizens are hypocrites, claiming to be for something, until it affects their own back yards. Or it could be, Massachusetts leadership is having \$100,000 spent on them, just in 2016.

## David Spiegelhalter on `how to spot a dodgy statistic’

In this political season, it’s useful to brush up on rhetorical skills, particularly ones involving numbers and statistics, or what John Allen Paulos called numeracy. Professor David Spiegelhalter has written a guide to some of these tricks. Read the whole thing. Highlights, though, of devices used to produce statistics which aren’t-quite-right (that is, wrong):

• Use a real number, but change its meaning
• Make the number look big (but not too big)
• Casually imply causation from correlation
• Use total numbers rather than proportions (or whichever way suits your argument)
• Don’t provide any relevant context
• Exaggerate the importance of a possibly illusory change
• Prematurely announce the success of a policy initiative using unofficial selected data
• If all else fails, just make the numbers up

David Spiegelhalter is the Winton Professor of the Public Understanding of Risk at the University of Cambridge and president elect of the Royal Statistical Society. Among many other things, he’s an advocate for expressing life risks as micromorts.

## Rushing the +2 degree Celsius boundary

I made a comment on Google+ pertaining to a report of a recent NOAA finding.

Enjoy.

But remember that COP21 boundary is equivalent to 450 ppm CO2.

## BOYCOTT natural gas, American or otherwise

It’s one thing to oppose pipelines and continued use of fossil fuels, but there is little as effective as a boycott of the key product. This is certainly not a new idea. (I don’t do Facebook. See this 2001 article as well.) So if you want to nudge in the direction of renewables, please consider boycotting natural gas. If you want to save money in the long term, please consider leaving natural gas. Natural gas and other fossil fuel prices are inherently volatile. Complaints of their being too high some times are really complaints about this volatility. Renewable energy produces electricity at the same price decade after decade.

Natural gas ain’t granola. Despite company advertisements to the contrary, drilling and fracking natural gas wells and associated infrastructure, including pipelines, compressor stations, and piping and metering stations are invasive, disruptive, expensive, and harmful to people, the environment, and the climate. Methane, the chief component of natural gas, is many times more powerful as a greenhouse gas than is CO2, and even at the burning end, combustion of natural gas is not complete, so there is leakage, even if the raw chemistry of all the component that is burnt is much cleaner than coal. Moreover, gas leaks, from pipelines, from nearly every step along the way, and especially in distribution networks near homes. And don’t think that because you hear reassuring things from utilities and gas companies and engineers that there’s safety there. It may be out of sight, but the political process and the Natural Gas Act of 1938 rigs the federal system against all opponents of natural gas, from cities and towns, down to localities and homeowners and farmers.

Co-constituents of natural gas with methane are carcinogenic and are powerfully harmful of human breathing and lungs. Even the odorants which are added to facilitate detection of leaks are themselves harmful.

• The easiest way is to design your new home with solar PV, energy efficiency, and electric heating/cooling in mind. Induction stoves are wonderful devices, bringing most of the benefits of gas stoves and energy efficiency to an all-electric footprint. Many new homes, particularly large ones, have excellent roofs and yards for solar PV arrays. Our home has appreciable tree shading, especially in summer, but we solved that by oversizing the array we installed, and, so, generating like crazy for the parts of the day and year we do see unimpeded Sun. And those puffy cumulus clouds are truly awesome helpers.
• Consider refitting your home and getting off natural gas. We were never on natural gas, but we once did heat our home with oil heated forced hot water and got our hot water that way. Now, our home is zero Carbon, since we heat and cool with ductless minisplits, and have an electric air heat pump hot water heater. We even have an electric, battery-powered lawnmower. Payback times are better each year. (Our solar array will pay for itself in 7 years, this being in Massachusetts.) Your state may have an incentive program and, from what I have studied, it is a win in any case: You’ll just not earn as much back as people do in states that have strong support of solar PV, like New York. The town of Minster, OH went big on solar despite their state’s punitive measures upon solar owners.
• If you cannot afford solar, or your roof or yard is shielded from Sun, or you live in an apartment, consider the Relay Power community solar program, and switch over from natural gas to at least one ductless minisplit system. Relay Power is available to anyone in the Eversource electricity region in Massachusetts, and there are community solar programs elsewhere in the United States. Check yours.
• Make your house more efficient! The less heat you need, the less gas you use. If you have a gas-powered clothes dryer, seriously consider drying your clothes by hanging them on racks on your deck or from a clothesline. Dampness in a house is not good for the house, and clothes smell better and feel better, in my opinion, when dried outdoors. If you use a gas oven, consider getting a much smaller top-of-counter electric oven. In our experience, most cooking for singles and couples does not need the big oven. Most of these small ovens are big enough to bake pizza. We have a Breville Convection Smart Oven and we love it.
• Support your local environmental organization and political action committee to push your state governor and legislature for better energy policy, one which advocates for zero Carbon energy. The more of this there is, the cheaper is electric power in your state, and the lower the price of natural gas, even if you continue to use it.
• Prohibit reimbursements like the Pipeline Tax. Utilities and gas companies are for-profit corporations. They should be able to get loans to build infrastructure from the private sector. They don’t need taxpayers to share the risk.

Natural gas … WE DON’T WANT YOUR PIPELINE We don’t want your damn gas.

## Mark Carney: Why are financial regulators and central bank governors looking at climate?

http://www.cbc.ca/i/caffeine/syndicate/?mediaId=725874755644

“We don’t want a Minsky moment about climate.”

Update, 2016-07-19

Interesting that Carney talks about “stabilizing at a temperature” when emissions are stabilized using a Carbon tax. He agrees with a Carbon tax, but he seems to have his science wrong. I did not get the impression he understands that to stabilize at any temperature, Carbon emissions need to go to zero. In his world, I wonder, does that mean that a price on Carbon needs to go to infinity? From my perspective, there is an implicit ceiling on Carbon price, and that is the realistic price per tonne to exact a unit of Carbon from atmosphere. Perhaps it would be more, before it’s not just about extracting this tonne of Carbon but this one, and another, and more. But, still, there is a kind of ceiling.

## JASA demands code and data be supplied as a condition of publication

The Journal of the American Statistical Association (“JASA”) has announced in this month’s Amstat News that effective 1st September 2016 “… will require code and data as a minimum standard for reproducibility of statistical scientific research.” Trends were heading this way, but it is excellent to see a major journal insisting upon it as standard practice.

There appear to be some weasel words allowing publications having “proprietary data” to move forward, insisting upon code nevertheless. I can only imagine that publications opting for that path will be seen as less established, solid, or compelling.

## Newt Gingrich and Van Jones. Right on.

It’s the thing. And it addresses how media and people forget about the actual statistics, and focus on the White Hot Bright Light.

What’s striking is that people have apparently forgotten that Statistics, as a field and a profession, was created principally as a vehicle for betterment of society. Not only was this true of creative originators like Florence Nightingale, but the very idea of creating mortality tables for mutual insurance companies.

## What makes me nervous

With regard to my comment at hypergeometric | July 13, 2016 at 3:50 pm on Tamino’s blog, someone challenged me on my assertion “Believe me, the +3C-+4C worlds are not places we want to go!” there. I have replied at Tamino’s blog, and I recommend reading the excellent comments there for context, but I thought it worthwhile to state my position here as well. Below is the quote. And we are apparently heading to the +3C to +4C region, even if COP21 is fully implemented. (See NCAR’s full report.)

While a detailed substantiation is inappropriate for this space, I’ll just mention three:

1. a World Bank report on the risks of a +4C degree world
2. The 2013 paper by Caballero and Huber indicating Eocene warming for atmospheric CO2 concentrations comparable to those we may see in the next 100 years and based upon paleoclimate records was +13C +- 3C warmer than today, not merely +4C
3. An introduction to and interpretation of Caballero and Huber by Ray Pierrehumbert (“Hot climates, high sensitivity”) in terms of non-linear, state-dependent climate sensitivity, described there as a Figure 1

The basis for my assertion is that we definitely do not want to approach the region [Professor] Ray [Pierrehumbert] writes “Here there (may) be dragons.”

While I am not a climate scientist, I am enough of a dynamicist to look at the rate with which we are introducing greenhouse gases compared to natural processes we can read in the paleorecord (with the possible exception of the Permian extinction event) to wonder whether or not we are actively exploring the climate state space for dynamical bifurcations. People who have examined the question suggest we would probably never know if we were approaching one. While there’s little that can be done except to press on for rapid reductions in CO2 emissions, I can only be honest and say these realizations make me very nervous.

Update, 2016-07-16

From Richard Pauli. See his blog for more.