# Category Archives: numerical software

## A look at an electricity consumption series using SNCDs for clustering

(Slightly amended with code and data link, 12th January 2019.) Prediction of electrical load demand or, in other words, electrical energy consumption is important for the proper operation of electrical grids, at all scales. RTOs and ISOs forecast demand based … Continue reading

## Sampling: Rejection, Reservoir, and Slice

An article by Suilou Huang for catatrophe modeler AIR-WorldWide of Boston about rejection sampling in CAT modeling got me thinking about pulling together some notes about sampling algorithms of various kinds. There are, of course, books written about this subject, … Continue reading

## A quick note on modeling operational risk from count data

The blog statcompute recently featured a proposal encouraging the use of ordinal models for difficult risk regressions involving count data. This is actually a second installment of a two-part post on this problem, the first dealing with flexibility in count … Continue reading

## Fast means, fast moments (originally devised 1984)

(Updated 4th December 2018.) There are many devices available for making numerical calculations fast. Modern datasets and computational problems apply stylized architectures, and use approaches to problems including special algorithms for just calculating dominant eigenvectors or using non-classical statistical mechanisms … Continue reading

## Cathy O’Neil’s WEAPONS OF MATH DESTRUCTION: A Review

(Revised and updated Monday, 24th October 2016.) Weapons of Math Destruction, Cathy O’Neil, published by Crown Random House, 2016. This is a thoughtful and very approachable introduction and review to the societal and personal consequences of data mining, data science, … Continue reading

## Repaired R code for Markov spatial simulation of hurricane tracks from historical trajectories

I’m currently studying random walk and diffusion processes and their connections with random fields. I’m interested in this because at the core of dynamic linear models, Kalman filters, and state-space methods there is a random walk in a parameter space. … Continue reading

## “Holy crap – an actual book!”

Originally posted on mathbabe:

Yo, everyone! The final version of my book now exists, and I have exactly one copy! Here’s my editor, Amanda Cook, holding it yesterday when we met for beers: Here’s my son holding it: He’s offered…

## Bayesian blocks via PELT in R

The Bayesian blocks algorithm of Scargle, Jackson, Norris, and Chiang has an enthusiastic user community in astrostatistics, in data mining, and among some in machine learning. It is a dynamic programming algorithm (see VanderPlas referenced below) and, so, exhibits optimality … Continue reading

## JASA demands code and data be supplied as a condition of publication

The Journal of the American Statistical Association (“JASA”) has announced in this month’s Amstat News that effective 1st September 2016 “… will require code and data as a minimum standard for reproducibility of statistical scientific research.” Trends were heading this … Continue reading

## data.table

R provides a helpful data structure called the “data frame” that gives the user an intuitive way to organize, view, and access data. Many of the functions that you would us… Source: Intro to The data.table Package

## On Smart Data

One of the things I find surprising, if not astonishing, is that in the rush to embrace Big Data, a lot of learning and statistical technique has been left apparently discarded along the way. I’m hardly the first to point … Continue reading

## Six cases of models

The previous post included an attempt to explain land surface temperatures as estimated by the BEST project using a dynamic linear model including regressions on both quarterly CO2 concentrations and ocean heat content. The idea was to check the explanatory … Continue reading

## Phytoplankton-delineated oceanic eddies near Antarctica

Excerpt, from NASA: Phytoplankton are the grass of the sea. They are floating, drifting, plant-like organisms that harness the energy of the Sun, mix it with carbon dioxide that they take from the atmosphere, and turn it into carbohydrates and … Continue reading

## high dimension Metropolis-Hastings algorithms

If attempting to simulate from a multivariate standard normal distribution in a large dimension, when starting from the mode of the target, i.e., its mean γ, leaving the mode γis extremely unlikely, given the huge drop between the value of the density at the mode γ and at likely realisations Continue reading

## Generating supports for classification rules in black box regression models

Inspired by the extensive and excellent work in approximate Bayesian computation (see also), especially that done by Professors Christian Robert and colleagues (see also), and Professor Simon Wood (see also), it occurred to me that the complaints regarding lack of … Continue reading

## November Hottest Ever, and Christmas Likely To bring Record Warmth in The East (Dan’s Wild Wild Science Journal; AGU Blogosphere)

The long-range guidance is showing strong indications that the incredible December warmth in the Eastern U.S. will continue to the end of the month. A blast of cold air will arrive later this week,… (Click on image for larger map, … Continue reading

## “1D Wave with Delta Potential and Triangle Initial Position” (Jeff Galkowski, Stanford)

The latest calculations from Jeff Galkowski, of Stanford.

## Southern Oscillation (SOI) correlated with Outgoing Longwave Radiation (OLR)

To the climate community this is nothing at all new, but I spotted these time series today and thought they would make a nice exhibit on how something people have direct control over, greenhouse gas emissions, affect a “teleconnection mechanism” … Continue reading

## Solar array with cloud predicting technology launched in WA

Australia’s first grid-connected solar power project with cloud predicting technology launched at Karratha Airport, WA, in bid to smooth solar supply. Source: Solar array with cloud predicting technology launched in WA

## Your future: Antarctica, in detail

Climate and geophysical accuracy demands fine modeling grids, and very large supercomputers. The best and biggest supercomputers have not been available for climate work, until recently. Watch how results differ if fine meshes and big supercomputers are used. Why haven’t … Continue reading

## Comprehensive and compact tutorial on Petris’ DLM package in R; with an update about Helske’s KFAS

A blogger named Lalas produced on Quantitative Thoughts a very comprehensive and compact tutorial on the R package dlm by Petris. I use dlm a lot. Unfortunately, Lalas does not give details on how the SVD is used. They do … Continue reading