See an important update.
No more walk.
I completely misjudged them.
See an important update.
No more walk.
I completely misjudged them.
I’m beginning a new style of column, called technical publications of the week. While I can’t promise these will be weekly, I will, from time to time, highlight technical publications I’ve recently read which I consider to be noteworthy. I am going to read them all again.
My professional emphasis, recently, for Akamai Technologies, has been on the plethora of adaptions of random projection methods (see also), generally based upon direct application of the Johnson-Lindenstrauss lemma or its several improvements. Many of these are collected under the rubric of locality sensitive hashing or LSH.
A first paper is called Earthquake detection through computationally efficient similarity search, and is by C. E. Yoon, O. O’Reilly, K. J. Bergen, and G. C. Beroza, and appeared in 2015 in Science Advances. It also has supporting online material. Using a technique for audio fingerprinting by Baluja and Covell, the authors develop fingerprints for earthquakes and convert these to signatures using LSH. These were used to assess classification accuracy of uncatalogued and catalogued earthquakes relative to a manually identified set for the Calaveras Fault in California, comparing performance to that obtained through the well-known but slower and more computationally expensive technique of autocorrelation, as well as the catalogue.
Yoon, O’Reilly, Bergen, and Beroza report very promising results, despite the great reduction in computation needed. Of greater interest to me is fitting the LSH into a larger signal processing task, including prefiltering and then interpreting results afterwards. They document the progress of a canonical data science project, offering the finished product, but strongly suggesting the pitfalls and backtracking they needed to undertake to bring it to success. That kind of experience is instructive for both students of data science, and the managers that expect results from these investigations.
Second, two papers applying LSH to health-related time series, with nice discussion of engineering tradeoffs for these applications:
Third, a paper, C. Luo, A. Shrivastava, “SSH (Sketch, Shingle, & Hash) for indexing massive-scale time series,” NIPS Time Series Workshop 2016, which offers an LSH-derived technique for preconditioning problems of time series comparison and lookups using dynamic time warping resulting in a net improvement of speed.
Fourth, not a paper, but an interview, from Dr Stephen Chu:
We heat and cool our home with Fujitsu `ductless minisplit` air source heat pumps. But this is New England, and it’s winter. A common question is how do they do under winter conditions?
Well, today we are treated to the increasingly rare Massachusetts blizzard, a nor’easter:
(You can see much larger versions of these images by clicking on them,
then using your browser Back Button to return to this blog.)
The storm producing this looks like this in EarthWinds:
It’s fed by excessively warm and moist conditions off the Northeast coast, themselves due to radiative forcing from fossil fuel emissions:
We have three minisplits. They are working fine. During a blizzard a little care needs to be taken to be sure snow does not pile up and cover the fan or the vanes which do the heat exchange, and sometimes these need to be brushed off. However, if the splits are operating at full, their built-in defrost does a pretty good job of taking care of this. They’ll continue to operate down to about -15℃ whereupon the heat pump will simply shut down. Below -12℃, efficiency drops markedly. This is a rare event, and is becoming increasingly rare. Right now, it’s about -8℃.
Should that occur, we crank up our now-off-and-orphaned oil furnace for the short time we need it, and shut it down when reasonable temperatures return.
And, should power from the grid go off — the power we’ve sent them during generation by our solar panels for the remainder of the year — our backup emergency propane generator kicks in, and can power essential elements of our house, including the oil furnace, but not the minisplits. Again that is a very rare event.
The generator (above) also needs snow kept clear from it.
By the way, here’s another shot of our beloved Rock Meadow Brook marsh, replete with Canada Geese, sheltering, and occasionally erupting in a burst of honks.
The photos above were taken just at the beginning of the blizzard. It’s good to have an after-the-fact set to compare, and to illustrate a feature of the Fujitsu minisplits. Here they are in their post-blizzard conditions.
Now, the heat exchange on the ductless minisplits happens when air passes over a fine array of thin fins at the backs of the units. The fans in the front draw air across the fins, and either heat is dumped into the air, for cooling, or extracted from the air for heating, as now. The feature of air source heat pump technology is that this heat can be extracted down to the -15℃ temperature mentioned above. All air temperature have some heat in the air, and it has nothing at all to do with differential temperature between the interior of the house and the outside.
However, in the case of inclement frosty weather, it is possible for snow to build up on these fins, interfering with their ability to exchange heat. Accordingly, because the Fujitsu units are designed to operate under such conditions, when it is present, they undergo a periodic defrost. You can see such frost on the fins of minisplit number 3:
I was fortunate enough to be outside when a defrost began on unit 3, and you can see the ice melting and puddling below the unit here:
Click on the image to see a close-up. (Use your browser Back Button to return to the blog.)
Minisplits 1 and 2 had already gone through such a defrost, and their fins were clear:
Naturally, this costs energy — electrical energy — but over the year it is not needed very often, and, so, the efficiency of the heat pumps, averaged over the year, remains a big win. Here’s a report from Sisler Builders in Vermont regarding their experiences.
I don’t have time to offer much in the way of explanation or comments here, but here’s the status of consumption per day, in Kilowatt-hours.
Update: I should have provided some context.
(Click on image to see a larger figure, and use browser Back Button to return to blog.)
Update, 2017-02-09: Google Earth images our solar panels
This is from May of 2016.
Painted signage on the side of a delivery truck parked outside a neighbor’s home deliverying oil made me curious about this, so I checked out their Web site.
I have heavily censored the image from the page to leave out the company name and trademarked references to products, and I will not provide a direct link to the Web site.
Note in particular the need for a reader of the page to be careful and critical with its statements:
What’s next? Environmentally friendly DDT?
Jane Lubchenco is a Professor at Oregon State University, and was administrator of the U.S. NOAA from 2009 through 2013, the U.S. Science Envoy for the Ocean at the State Department from 2014 to 2016, and the president of the Ecological Society of America from 1992 to 1993. She recently wrote about her current perspectives on environmental science. Here are excerpts.
… Just when, thanks in part to US leadership, the world finally began to make tangible progress in addressing climate change, the US elected a President who labeled climate change a hoax and whose Cabinet nominees leave little doubt that climate denial will continue. Equally problematic are the blatant disregard of facts and lack of respect for others and for civil discourse that were painfully evident in the US elections and around the world. So pervasive was the dismissal of “truth” that the Oxford English Dictionary named “post-truth” as the 2016 “Word of the Year”, defining it as “relating to or denoting circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief”
[T]ake heart! I believe we can rise to this occasion with the boldness, energy, and creativity it demands. Not in a knee-jerk fashion, but one that responds to some of the underlying causes of our current dilemma. We must engage more vigorously with society to address the intertwined environmental and social problems that many have ignored, to find solutions, and to help create a better world. We must truly listen to and address the reasons why a post-truth world has emerged.But we cannot do so from lofty perches above society; we must be more integrated into society. It is no longer sufficient for scientists in academia, government, nongovernmental organizations (NGOs), or industry to conduct business as usual. Today’s challenges demand an all-hands-on-deck approach wherein scientists serve society in a fashion that responds to societal needs and is embedded in everyday lives. Humility, transparency, and respect must characterize our interactions.
Dan Kammen of U.C. Berkeley says more along these lines:
I received a link to this letter regarding the 27th January 2017 White House Executive Order on visas and immigration from the American Meteorological Society. I am also a member of the American Statistical Association, the Ecological Society of America, the American Association for the Advancement of Science, and the Institute of Electrical and Electronics Engineers.
Except for the IEEE, which is still trying to figure out what to do, each of these organizations signed this letter. [Added 2017-02-02.] The IEEE issued a separate statement later.
And, the IEEE chimes in:
IEEE President Karen Bartleson today released the following statement in response to concerns expressed by IEEE members around the world:
“IEEE, The Institute of Electrical and Electronics Engineers, Incorporated, believes that governments of all countries must recognize that, in a world of increasing global connectivity, science and engineering are fundamental enterprises, for which openness, international collaboration, and the free flow of ideas and talented individuals are essential to advancement.
“Every country benefits from attracting, and competing for, the best and brightest scientists and engineers from around the world to study, teach, conduct and collaborate on research, innovate new technologies, and start commercial endeavors. Science and engineering lead to enhancements in quality of life and ultimately build economic prosperity and security. All countries should develop and maintain immigration and visa policies that encourage, facilitate, and protect the ability of people, from around the world, to engage in these types of science and engineering activities.
“Diversity is an important and valued strength; IEEE is committed to the realization and maintenance of an environment in which scientists and engineers, regardless of ethnicity, religion, gender, or nationality, have the right to pursue their careers without discrimination. Science, engineering — and humanity — prosper where there is freedom of movement, association, and communication.”
On January 27, 2017, the President signed an Executive Order regarding immigrants and refugees from certain Muslim-majority countries. The order has now been challenged in a number of jurisdictions. As the Acting Attorney General, it is my ultimate responsibility to determine the position of the Department of Justice in these actions. My role is different from that of the Office of Legal Counsel (OLC), which, through administrations of both parties, has reviewed Executive Orders for form and legality before they are issued. OLC’s review is limited to the narrow question of whether, in OLC’s view, a proposed Executive Order is lawful on its face and properly drafted. Its review does not take account of statements made by an administration or it surrogates close in time to the issuance of an Executive Order that may bear on the order’s purpose. And importantly, it does not address whether any policy choice embodied in an Executive Order is wise or just. Similarly, in litigation, DOJ Civil Division lawyers are charged with advancing reasonable legal arguments that can be made supporting an Executive Order. But my role as leader of this institution is different and broader. My responsibility is to ensure that the position of the Department of Justice is not only legally defensible, but is informed by our best view of what the law is after consideration of all the facts. In addition, I am responsible for ensuring that the positions we take in court remain consistent with this institution’s solemn obligation to always seek justice and stand for what is right. At present, I am not convinced that the defense of the Executive Order is consistent with these responsibilities nor am I convinced that the Executive Order is lawful. Consequently, for as long as I am the Acting Attorney General, the Department of Justice will not present arguments in defense of the Executive Order, unless and until I become convinced that it is appropriate to do so.
Acting Attorney General Sally Yates was fired on 30th January 2017. I applaud her.
(Updated the afternoon of 3rd February 2017.)
Give what you can. Thanks!
This is all about The Right to Know.
Update, 2017-01-20, 18:11 EST
Also see an article in MIT’s Technology Review.
Update, 2017-01-20, 21:19 EST
Update, 2017-01-25, 0102 EST
We are now in active
Internet information war.
Update, from 0400, 25th January 2017
From Reuters (hat tip to Benjamin Rose):
Trump administration tells EPA to cut climate page from website: sources
U.S. President Donald Trump’s administration has instructed the Environmental Protection Agency to remove the climate change page from its website, two agency employees told Reuters, the latest move by the newly minted leadership to erase ex-President Barack Obama’s climate change initiatives.
The employees were notified by EPA officials on Tuesday that the administration had instructed EPA’s communications team to remove the website’s climate change page, which contains links to scientific global warming research, as well as detailed data on emissions. The page could go down as early as Wednesday, the sources said.
“If the website goes dark, years of work we have done on climate change will disappear,” one of the EPA staffers told Reuters, who added some employees were scrambling to save some of the information housed on the website, or convince the Trump administration to preserve parts of it.
The sources asked not to be named because they were not authorized to speak to the media.
A Trump administration official did not immediately respond to a request for comment.
The order comes as Trump’s administration has moved to curb the flow of information from several government agencies who oversee environmental issues since last week, in actions that appeared designed to tighten control and discourage dissenting views.
The moves have reinforced concerns that Trump, a climate change doubter, could seek to sideline scientific research showing that carbon dioxide emissions from burning fossil fuels contributes to global warming, as well as the career staffers at the agencies that conduct much of this research.
The page includes links to the EPA’s inventory of greenhouse gas emissions, which contains emissions data from individual industrial facilities as well as the multiagency Climate Change Indicators report, which describes trends related to the causes and effects of climate change.
Update, 2017-01-25, 14:45 EST: We mattered
See also the reports of an underground EPA.
(Click image for a larger picture. Use browser Back Button to return to blog.)
Hat tip to Climate Denial Crock of the Week for image.
We have our first evidence of Web site alteration, at the EPA.
Also, here are links to some recent articles about this effort and its context:
Hat tip to Professor John Baez.
(This blog post was updated 19th January 2017 with a correction to the interpretation of the leak data. The correction was offered by Professor Phillips. The blog author is responsible for the original misunderstanding. Apologies for any inconvenience.)
The West Roxbury Lateral (“WRL”) has long been the subject of protest, opposition, and conflict regarding the roles of fossil fuels, particularly natural gas, in providing electrical energy and heat to the Commonwealth of Massachusetts.
Well, Professor Nathan Phillips of Boston University who has had a major role mapping natural gas leaks in pipes in Boston has recently imaged a portion of the newly built WRL, this part at its entry into the Town of Dedham. He has produced the following image, and released it on Twitter:
Note the figure does not show the amounts quantitatively, but clearly as the pipeline goes on to the east, the baseline leakage tracing its path is “normal.” The leaky section clearly is not.
Professor Phillips submitted the following comment regarding this data:
I just have one clarification, which is that we aren’t making the claim that the WRL pipeline itself is leaking, but that there are leaks along its route, likely from the existing low pressure distribution system. But we don’t know for sure. The issue is that there is new pavement and sidewalks overlying old leaking pipes – a huge wasted opportunity and already, a new roadway is punctured by patches.
And he offered this record of the data gathering:
This is the route of a brand new transmission pipeline, which only went into service a month ago or less. As Professor Phillips says this was a “missed opportunity“ because, while the trench was open, the companies should have taken the opportunity to repair existing pipelines.
By Carl Safina.
I have made an important update to an earlier post here, Getting back to 350 ppm CO2: You can’t go home again.
The message, essentially based upon recent work Tokarska and Zickfield on one hand, and by The Global Carbon Project on the other make the calculation of geoengineering through clear air capture of CO2 far more pessimistic than even the whopping cost numbers were before. (Thanks to Glen Peters for pointing me to the annual budget of Carbon compiled by the Global Carbon Project.)
See the post, but, in short, I forgot to account for CO2 “dissolved” in oceans and terrestrial ecosystems which will come back into atmosphere in order to restore pCO2 equilibrium once atmospheric concentrations are reduced.
You really can’t go home again ….
This all also reminds me of something I have written before. Forget the monuments, and the civilization, and the going to the Moon or Mars, and the libraries. In the long run, humanity’s legacy to Earth and the Universe will be the Carbon Dioxide we are emitting into the climate system. No other single action we can imagine doing in the foreseeable future will have such a widespread or a long-lasting impact. Right now, we are Carbon Dioxide.
There is an excellent piece in Ars Technica about why scientific measurements need to be adjusted, and the implications of this for climate data. It is written by Scott K Johnson and is called “Thorough, not thoroughly fabricated: The truth about global temperature data.”
Mr Johnson writes:
… In fact, removing these sorts of background influences is a common task in science. As an incredibly simple example, chemists subtract the mass of the dish when measuring out material. For a more complicated one, we can look at water levels in groundwater wells. Automatic measurements are frequently collected using a pressure sensor suspended below the water level. Because the sensor feels changes in atmospheric pressure as well as water level, a second device near the top of the well just measures atmospheric pressure so daily weather changes can be subtracted out.
If you don’t make these sorts of adjustments, you’d simply be stuck using a record you know is wrong.
This is the kind of thing that’s learned in Physics and Chemistry classes in high school these days. (Well, at least AP Physics and Chemistry, not to mention Statistics.)
Mr Johnson provides a nice sketch of the several datasets use to estimate Earth surface temperature data. There’s a similar story which attends sea-surface temperatures, which has its own dramas, also describe here, from water inadvertently heated by ship’s engines, which Mr Johnson mentions, to thermal bias and microcode errors in measurement instruments.
These are experimental adjustments, made for good reason. There are also statistical adjustments which can improve representations of datasets, like smoothing, which I have written about earlier.
But the point is, many people, encouraged by a sound-bite-oriented media, don’t know about or understand these complications, and so it is easy for people like Representative Lamar Smith to prey on their ignorance. Is it his fault? Partly. But it’s also the fault of a public which embraces representative democracy but doesn’t “want to go to school and learn their lessons” well enough to be able to fulfill their responsibility.
Interesting that Dr Schmidt has some gentle criticism of the PBS program NOVA.
Whole talk is here:
This was posted in February 2015.
It’s heading towards year’s end, so it’s natural to think about perspective.
In a post from last July, Joseph Heath asks semi-rhetorically, “Why are [proposed] carbon taxes so low?” and, then, he and commenters go on and answer that, essentially, the cost of damage is discounted to the present to obtain estimates of the Social Cost of Carbon.
Except that none of these methodologies I see incorporate the full cost of perhaps someday needing to not only decarbonize but to do clear air capture of carbon dioxide and sequestering it (effectively) permanently. They are making estimates of damage from climate disruption.
However, the costs of clear air capture includes an up-front cost of decarbonizing first, since capture is doubly and triply more expensive if we continue to pollute.
See my blog post where the estimate for one such exercise puts the price at US$1800 trillion in constant 2010 dollars. Even if not only is there no inflation in the price, but one additionally applies a discounting rate of 4%, after 100 years that’s still US$3.6 trillion. Worse, the scenario is sensitive at when we start, where we want to reduce to, and whether or not emissions are first zeroed, let alone invoking a technology we do not yet have. This is why my view has now aligned strongly with those of Glen Peters and Kevin Anderson.
I also, and personally, am very pessimistic about the wealthy nations (OECD) of the world doing enough, fast enough, in changing their behavior and their economies to make a significant difference. And I say that despite being extremely enthusiastic about the potential of zero Carbon energy technology, especially solar.
The OECD countries will eventually do it, because:
What kind of substantial things am I doing?
And, while I will engage with people online and elsewhere stating and shouting incorrect things regarding the environment, or climate science, or zero Carbon energy, or who is responsible for all this, I am disengaging emotionally, because it does not matter. Science and engineering facts, on the other hand, do matter, and are worth defending with some ferocity. These are the only hold we have on reality, as opposed to a so-called reality TV show.
And I am sorry that the people who, at least initially, are being hurt and harmed by climate disruption are people who have the least responsibility for the problem. I cannot control people in my world so they begin shed the behavior sets responsible. I can entice them with the wonders and efficiencies of zero Carbon energy. And while climate justice is important, I fear it can be counterproductive compared to, say, campaigns to boycott use of fossil fuel energy. Pursuing climate justice might be simply a nice, and very white way of soothing piqued consciences, and stops further progress where it is more important.
Whether it is to be Utopia or Oblivion will be a touch-and-go relay race right up to the final moment…. Humanity is in ‘final exam’ as to whether or not it qualifies for continuance in Universe.
Hat tip to … And Then There’s Physics for the motive to write this post.
From Katharine Hayhoe, who I deeply respect, and from John Cook (*), scientists and the quantitative community have been scolded that the reason they don’t make headway with the public and the science denier community is because their explanations are too quantitative, that they are too wrapped up with physical processes and models, and mechanisms. Instead, some of these experts at communication argue, stories should be told, which reach across what was once called the two cultures divide.
Yet, even when that is pursued, voices familiar with the quantitative and with what, to them, are sounds shrieking danger at its highest (Hosanna!), find themselves adrift in a murky sea of counter-stories.
Nothing. They will not listen.
And, to me, the only remaining event which people might pay attention is if, on otherwise fine days along the East coast of the United States, people with expensive properties, in Miami Beach, in Boston, in the Carolinas and Maryland, find those properties suddenly awash in salty water, twice a day, and their property values rushing towards zero, beyond rescue of insurance, or FEMA, or the Biggert-Waters Act.
Then, as the great doctor Neil deGrasse Tyson points out, when people with wealth begin losing that wealth, they may want to pay attention.
Dr Tyson’s patient waiting is where I feel I am these days. I am tired of trying to communicate to people who just don’t want to listen, even if if they know how.
And, then, there’s the vital, defiant spirit of Governor Jerry Brown, of California:
One of the vices in the spiritual life is called tepidity. We’ve had a lot of tepid climate fighters, people who are not really telling the full truth. But there is a paradoxical benefit when someone takes to an absurd length a completely erroneous position. That so unmasks the error that it allows everyone else to refute it.
I’m not discouraged. I can’t think of anywhere else to be than in climate science today. Fights are fun. And this fight is big. And it’s gonna be attractive, and it’s gonna take a lot of smart people.
(*) I took Dr Cook’s course. It was fine as well as it went. But it’s suggestions for moving forward were, politely put, anemic.
Just about a year ago, our home in Westwood began a march towards zero Carbon consumption, with heating, hot water heating, and even lawn mowing all converted to high efficiency electricity. As indicated at the time, our main automobile, a Toyota Prius, remained the major obstacle. We also have an old 2005 Toyota Corolla which is used, basically, to drive me 5 miles (tops) to the train station (and back 5 miles) for the 2-3 days I go into the office for work. (I work from home the other 2-3 days.)
Today, Claire took delivery of a leased 2017 Chevrolet Volt LT Hatchback, from Muzi Chevrolet in Needham Heights, with Rob Roderick providing the excellent introduction and service. Seth Fletcher of Scientific American calls the Volt “impressively unremarkable.”
The Volt is a circuit board with electrical motors mounted atop, having a gasoline-powered electrical generator under its hood. It has all the pep you’d expect of electrical drive, and the generator helps navigate a region which is yet sparse in charging stations and a limited 60-80 mile range. The Chevy Bolt promises to be better, if a bit more expensive.
Ours is a 39 month lease. We’ll see what time brings.
Thanks to Paul Lauenstein for the tip.
Owners manual for the curious.
When knowledge conquered fear …
And, what better way to celebrate than watching the National Geographic Cosmos episode, When knowledge conquered fear, hosted by the great Dr Neil deGrasse Tyson, Director of the Hayden Planetarium in New York City.
The National Center for Atmospheric Research (“NCAR”) reports on a newly substantiated teleconnection between positive sea surface temperature anomalies (“SSTA”) in the Pacific and the temperatures over the continental United States (“CONUS”) 50 days later. A teleconnection is:
A linkage between weather changes occurring in widely separated regions of the globe.
as defined by the American Meteorological Society (of which I am a member).
The basic evidence (but see the NCAR post is in the following figure:
(Click on image to see larger figure, and use browser Back Button to return to blog.)
That’s modest warming. The teleconnection predicts that will influence temperatures in CONUS in the second week of February, 2017.
Those are hot. And you’ll note that large dark blue blob at the top right … A cold Arctic outflow. That stops the Gulf Stream flow northwards, slows it down, and piles the waters up against the Northeast, contributing to sea level rise there.
And they who will not be ready, will suffer the economic consequences.
(That water is a foot deep, previously reported in a post here. Click on image to see larger picture, and use your browser Back Button to return to this blog.)
Both articles from the great Ars Technica.
More links to and comments on the same event:
- San Francisco Chronicle
- From the office of California’s governor
- From Climate Denial Crock of the Week
- On the Carbon Lobby and the Trump Gang
An interesting discussion of what a Trump administration could and could not do to repositories of climate data. The possibility of this kind of thing, and retaining control of data provenance and the sequence of transactions done to data is a reason why different flavors of trusted timestamping might be a good idea for all these sources, including blockchain techniques. It’s not clear if the courts and legal systems are up to trusting them yet. They do trust private-public key cryptography, due to changes in law. But Science might trust them.
And I think the various kinds of manipulations that geophysical and oceanographic data are subjected would pose a challenge to archivists and blockchain technologists, particularly when large sets are combined using a specific algorithm embodied in code to produce a result. It seems to me the chain of the code needs to be joined in the chain, too.
The quotation is portrayed at the very end of the 1970 film Tora! Tora! Tora! as:
I fear all we have done is to awaken a sleeping giant and fill him with a terrible resolve
Snapshots from the American Geophysical Union Fall Meeting.
Above, California Governor Brown gave a rousing and defiant talk that was well received.
Below, updated animation from the Arctic Report Card.
The year showed “a stronger, more pronounced signal of persistent warming than any other year in our observation record,” he said.
These changes have had considerable impacts…
View original post 639 more words
Your CO2, my CO2 doesn’t remain with you or me, but mixes broadly and thoroughly over the planet at large.
So, we all share responsibility for the damage.
Credit: NASA And brought to you by OCO-2.
David Puttnum (yes, the producer-director) has a very moving appeal on climate:
Hat tip to Tamino.
President Lyndon Johnson was the first to receive a briefing regarding the looming crisis presented by abrupt climate change. That was in 1965. And we’ve been “waiting for more substantial evidence“ ever since.
It’s pretty clear some people’s wishful thinking means they won’t change their minds until until they witness a sufficiently severe and unprecedented natural demonstration, and the threshold will differ. Unfortunately, by the time this happens or they succumb to acceptance, doing something about it will be horrifically expensive, and will be all the more expensive because wealth will be being destroyed by these changes in anything but a gradual manner.
My advice to kids? Get the most education, the best training you can, in the deep sciences, or engineering, or especially mathematics or medicine, and insulate yourself as much as you can.
To the aware us? We need to continue to fight for a human future.
And to the people who doubt, deny, delay, profit … you won’t have to answer to me: Nature will take your wealth, whether from you, or from your children. But you do have a lot of forgiveness to ask, of the millions around the world who suffer because of your pursuit of comfort.
Behold, this was the sin of your sister Sodom: she and her daughters had pride, surfeit of food, and prosperous ease, but did not aid the poor and the needy.
Once nice thing about having a net metered solar PV array is that, with a little diligence, you can figure out how much electricity your household is consuming each day, or at finer resolution if you like (*). Below is the report of that from mid-May through early December 2016 for ours. Recall we have a zero Carbon home, but no EV (see also). We heat and cool the house with air source heat pumps (“ductless minisplits”) as well as heat hot water with air source heat pump and sometimes hybrid electric element (50 gallon GE GeoSpring hot water heater, with hybrid element being a more efficient way at times of using electric energy).
This is the consumption as calculated by the change in meter readings day-over-day added to solar generation. Change in meter readings can be negative when we give back to the grid more than we use, and that’s when we earn towards a snowy or rainy day. Not all days were recorded, but when they were not the total energy consumed and generated in the gap was tallied, and an average energy per day used to impute the consumption.
The Thanksgiving Day spike is pronounced. Not a surprise: Claire did a bunch of baking, and roasted a turkey breast for guests. (We’re vegetarians.) We have an electric induction stovetop, and a conventional big oven. We typically use a small convection oven and our microwave really gets used a lot.
We have a dishwasher which is run nearly every day, and we shower every other day or so, depending upon our athletic exercise schedules.
Our lights are now mostly LEDs with a couple of lamps still using fluorescents. We do put up a small number of holiday lights.
Our washer and dryer are electric, and most of the year Claire air dries clothes on our deck, using the dryer for fluffing.
I’ve encountered a number of blog posts this week which seem not to understand the Bias-Variance Tradeoff in regard to Mean-Squared-Error. These arose in connection with smoothing splines, which I was studying in connection with multivariate adaptive regression splines, that is actually something different than smoothing splines. (I will have a post here soon on multivariate adaptive regression splines, or the earth procedure as it’s called.)
The general notion some people seem to have is that smoothing splines throw away information and introduce correlation where there isn’t any, and it distorts scientific data. A particularly obnoxious example of this is at science denier William Briggs’ blog. Another, milder instance is at a blog post by a blogger called “Joseph” who specializes, he says, in “A closer look at scientific data and claims, with an emphasis on anthropogenic global warming.” I was going to put in a comment at the blog, but apparently comments there are closed, or at least no longer work. (So do some links to data from that post.) So, instead, I’m putting it here. I already answered a question at Stats Stackexchange which invoked Briggs.
Smoothing is not about making a picture nicer or losing information. It is about the bias-variance tradeoff. Given that minimizing mean squared error in fitting data with a non-parametric (or, for that matter, any) model is important, introducing a bias in a model, such as smoothing in a spline can reduce variability and, so, reduce overall mean squared error of a fit.
The Wikipedia page shows the connection with bias and variance, and the proof of their relationship.
It was an important finding by Stein in 1955, which gave rise to deliberately introducing some bias via things like James-Stein estimators in order to improve overall performance. Prior to Stein’s insight, classical statistics only considered unbiased estimators, and that insight showed that procedures like maximum likelihood estimation were not optimal, even if they work well a lot of the time.
And, accordingly, “Joseph”‘s criticism of the Law Dome CO2 data is not well founded. I bring his and the reader’s attention to a paper co-authored by Etheridge, one of the co-authors of the Law Dome work, about why smoothing splines are used.
Note mean-squared-error is disguised in various powerful measures of model fit, like the Akaike Information Criterion.
Update, 2016-12-27: Smooth, yes, but don’t ever expect to see the smoothed curve realized
While the smoothed version of a series can and often does provide an estimate with the least mean-squared-error, if properly chosen, it is a different question whether the presentation of such a smoothed curve is the best to convey the series, especially if communicating with the statistically uninitiated. The smoothed version of a curve is an idealization, intended for purposes of forecasting, or prediction (they are not the same), and sometimes for helping to tease out physical mechanisms giving rise to the observed phenomenon.
For one thing, the smoothed or idealized curve has zero probability of actually being realized, even on the span of support for which it is calculated. Actual realizations of the phenomenal or observed series will have excursions from the smooth guided by the distribution of its residuals, and it is entirely a part of the series to see these excursions and, moreover, expect that if (it were possible to draw) another realization of the series, there would be a different set of excursions applied.
For another, the general public does not seem to get the idea of a data series with random excursions atop a pattern, and appear to approach these matters as if they were entirely deterministic. That’s a very classical kind of notion: The Watchmaker’s Universe. In this view, the only reason why phenomena are not perfectly predicted is because we have but imperfect knowledge of the science involved, or of Nature, or something, and only a Deity knows these (notwithstanding the Deity knowing what all individuals will choose if Free Will is posited). A different view, more modern is that even a Deity cannot predict perfectly how another realization of these stochastic phenomena will play out.
So, the best way to communicate this variability to me is to present the observed data from the series, present the smoothed realization, and then present a cloud or ensemble of draws from the smoothed curve with excursions governed by residuals atop of it. For example,
(Click on image to see larger figure, and use browser Back Button to return to blog.
By the way, the example above shows two competing models for the smooth to the data.
If dependent data are to be emphasized, then using ensembles of tracks such as the reasonably famous hurricane tracks are useful:
(Click on image to see larger figure, and use browser Back Button to return to blog.
Topics in Computational Neuroscience & Machine Learning
Lecture notes for CPSC 536N "Randomized Algorithms"
Random thoughts on Machine Learning, Deep Learning and (sometimes) Computer Vision
stories we tell ourselves
Research that Informs Business and Public Policy
Guyana News and news from Guyanese Associations worldwide
"For here we are not afraid to follow truth wherever it may lead, nor to tolerate any error so long as reason is left free to combat it." – Thomas Jefferson
Energy, Environment and Policy
likhipa inhlanzi emanzini
... for when you can't solve life's problems with statistics alone.
Exploring and venting about quantitative issues
Newscasts on Global Warming, Its Consequences & Solutions
A Welcoming Congregation & A Green Sanctuary
Stop the war; stop the warming.
e-commerce data science and analytics
Boston Area Sustainability Group
with Peter Sinclair
I can calculate the motion of heavenly bodies but not the madness of people. -Isaac Newton
Engineering, Oceanography, and Innovation in Environmental Science
Changing the world and its people, one post at a time
notes on mathematical finance, algo trading and derivatives
Discover change, together
Tips and tricks on programming, evolutionary algorithms, and doing research
Experiments & Experiences in R
Critical perspectives on technology, sustainability, and the future
Astronomy, space and space travel for the non scientist
How one atheist sees life
The Forum for Climate Engineering Assessment: Unpacking the social and political implications of climate engineering
Noah Deich's blog on all things Carbon Dioxide Removal (CDR)
resources on approximate Bayesian computational methods
The Science, Economics, and Politics of Climate Change
Multa novit vulpes
Once you have finished counting diatoms, the real fun begins
weather musings for a wider audience
Let's talk about how we do research in the weather and climate sciences
science and current events d1p
``Your goodness must have some edge to it,--else it is none.'' R. W. Emerson
To inform, inspire and involve
FastMail system status announcements
Science, Politics, Life, the Universe, and Everything
Ponderings of science, philosophy, history, politics, and many other topics
Statistical Computing + Bayesian Modelling
Science - Simplified
A Photographic Journey