Monday, December 15, 2014

Pfafstetter Numbering and the organisation of river networks

We just have accepted a paper related to the topological ordering we use inside our model JGrass-NewAGE.  This ordering derives is a generalisation of the Pfafstetter algorithm.
Once understood, it can be observed that this Pfafstetter ordering can be used to drive, for instance parallel execution of operations. But this is another story. We actually use to orderly process the basin partition in JGrass-NewAGE modules. The paper, entitled "Digital watershed partition within the NewAGE-Jgrass system" by Formetta et al.  can be downloaded from here.
Related topics are covered in posts of the Horton Machine, and particularly in the book Chapter written early this year for the British Gemorphological society. Abera's blog also contains a related post that can clarify some other issues. 

Friday, December 12, 2014

Using geostatistics to integrate satellite information and modelling on soil moisture

This paper has a long history and explore the idea that geostatistics can be used to integrate satellite information when this is missing. At the same time the whole information is used for assimilated for better driving the Community Land Model. Thank you to Han Xujun for pursuing the publication, when I abandoned any hope, notwithstanding that the paper is a good one.

The paper is entitled: Soil Moisture Estimation by Assimilating L-Band Microwave Brightness Temperature with Geostatistics and Observation Localization, and my co-authors are (in order):
Han Xujun, Xin Li, Rui Jin, and Stefano Endrizzi.  The paper has been accepted by PLOSONE, and you can find the pre-print  here.

Other papers by Xujun are available from his Research Gate Profile.

A little of further bibliography:

Han, X. J., et al. (2014). "Soil moisture and soil properties estimation in the Community Land Model with synthetic brightness temperature observations." Water Resources Research 50(7): 6081-6105.

Han, X. J., et al. (2013). "Joint Assimilation of Surface Temperature and L-Band Microwave Brightness Temperature in Land Data Assimilation." Vadose Zone Journal 12(3).

Han, X., et al. (2012). "Spatial horizontal correlation characteristics in the land data assimilation of soil moisture." Hydrology and Earth System Sciences 16(5): 1349-1363.

Saturday, December 6, 2014

Imagine to be a hydrologist and you want to learn Java

Dear * to learn Java: 

you can use my lectures (still being produced):

Read the books I list in my blog: Many of them are specifically dedicated to Numerics and Scientific Computation.

Learning Java is very much programming, not just reading. So You have to choose a task and try to perform it. Good experiments should be, creating reusable classes for:
  • Reading and writing data to a file
  • For a generic function
  • For solving an Ordinary differential equation (or a set of them: Lorenz' chaotic equations would be a good exercise indeed

If you do math, sooner or later you to use matrixes. You do not need to reinvent the wheel, even if a standard choice has not been yet emerged. See here.

In general all the Java resources I came across (including this one, are at):

Francesco Serafin, master thesis, introduces to various tools and methods that can be used to integrate partial differential equations:

The subsequent step is to learn OMS: maybe this can be a little more complicated. Anyway, I way to do this is to start with the 2013 summer school:

or you can go and work out the examples you can find at the original OMS site:

I will be improving the resources all the time. So check them once in a while !

Tuesday, December 2, 2014

Evapotranspiration parameters in coarse grained modelling

Let me have a little rehearsal on Evapotranspiration (see my posts for "random" thinking on the topic) estimation with simplified model, like Penman-Monteith or Priestley-Taylor. Literature reports the concept of "Potential Evapotranspiration", ETp, but, as Brutsaert says, the concept is a slippery one. From the conceptual point of view, from what I understand, it is the ET when no resistance is acting, except what is implied by limitations by atmospheric forcings. One can try to see it in the field or deduce from formulas. In Penman-Monteith (PM) approach:

this has a clear interpretation, which is, the ET you obtain by putting any resistance, rg (either representing soil or vegetation, indeed), to zero.

Unfortunately this is not the same in Priestley-Taylor (PT), where the resistance terms simply are not present:

where everything is lumped in the alpha coefficient. What people did in this case, I think, to assess alpha in ETp (say alphap) was to measure ET in conditions supposed to be those in which ETp realizes and estimate the alphas which are reported in literature (IMHO this is the only way it can be accomplished but I confess I did not went into the detail of that literature, see for instance the overview by Cristea et al, 2012). Once your get alphap you can estimate ETp but still you have to introduce a further reduction to get the real ETa. The method popularized by the ecohydrology literature (e.g. read Amilcare Porporato here) is to introduce a linear decrease of ETp with water storage in the root zone "reservoir".

Both the passage, the determinatio of ETp in the framework of PT and the linear reduction with storage have, in my view, strong drawbacks from the quantitative point of view.
One can get the alphap, but literature show a huge variability. So literature is quite useless to obtain quantitative results, ith a decent ncertainty.
The (linear) decrease of ET with soil moisture requires the determination of at least one additional coefficient. In fact, it is well known that ET has two stage: stage one, when ET is "at the potential rate", independently from the water content up to a critical soil moisture, well below saturation, when ET is depressed, not by increasing suction (the so called Kelvin effect, which is a second order effect) but by the fact that pores at the soil or leaf surface to which water is supplied are more and more far apart (see recent literature by Dani Or and co-workers). This critical soil moisture, at which the second stage ET starts is a further coefficient, and its identification with saturation implies a clear underestimation of ET. It is usually given for granted by my friends ecohydrologists and my master IRI's literature that it can be determined. But I do not have to remind to you all how much elusive it is the definition of the "root zone soil moisture" just to cite a practical aspect of it.
Even if field-fellow-scientists claim to have measured it, I know that who tried in lab of few square meters really struggled to close the water budget budget under very controlled conditions (let's say: personal communications). In nature, as my hero Pete Eagleson teaches, interaction among plants distribution, atmosphere, and rugged terrain makes any of the above coefficient heterogeneous, and the trials to find a rational to all of it, kind of frustrating to my eyes.

Said all of this, let's go back to PT, and its use conjointly with a rainfall-runoff model. Simplifying a little the PT formula reads:

Where alpha is the PT coefficient, f(theta) a function os soil moisture content, F(T) = Delta/ (Delta + gamma) has a weak dependence on temperature T, Rn the net radiation. (The latter, in our systems is estimated by properly accounting the topographic variability, while in most of applications is given by a rough estimate ... ). Please observe that there is no way to get rid of the alpha coefficient. So even if we would implement a way to get into account of f(theta), for instance introducing a linear decrease with soil water content, we still have to estimate alpha.
The more direct way we have for doing it, is, in my and Wuletawu Abera opinion, using the water budget, but I will come back to this in another post.

A bare-bone formulation of f(theta), is it being either 1 if there is enough water in the soil reservoir (in whatever way we account for its content in our modelling) or cut it to the maximum amount of water available when there is not enough. This is, for instance the strategy used by the Hymod model, when working at daily time scale. Actually, to be sincere, this limitation is reasonable in the original formulation of Hymod (e.g Formetta et al., 2011), working at the daily scale. However, I am sure is not limiting anything at hourly scale in which, for instance our Hymod implementation can work. This to say, that, in our formulation f(theta) should be essentially always one. 

In principle, if, for a reason or another, we leave f(theta) =1, we can evaporate also if there is no water in soil. But what we can do if we have at most one reservoir for hillslope, or in many cases where Hymod is used, just one reservoir for catchment to account for soil misture ? This simplification itself hide the complexity of soil moisture distribution at sites scales (and we know that convex-divergent sites are potentially less wet than convergent-concave, and possibly very close to be completely dry in summer). To be realistic we should invent some further function that says which is the distribution of wetness in the catchment, given the total amount of water, and we need to invent to map a distribution into a mean reduction of ET at catchment scale .... but this is not anymore a bare-bone solution.

The often used linear dependence of f( ) on theta, is not "scale invariant". In fact, assuming it exists for any site, when coarse graining the physics at hillslope (HRUs) scale, it must coming out from integration of sites evapotranspiration over the whole space.

If we can assume that the alpha is a space-time constant (and therefore we can take it out of the integral, certainly this is not for both soil moisture content and radiation, which are known to be highly variable in space. Making the linear assumption valid again, means therefore to assume that

for some representative (space) "mean" (with bars) values of soil moisture and radiation content. This statement is certainly true, if we assume space continuity of both the quantities (given for granted for radiation, but less granted for soil moisture), but to be empirically determined case by case for any practical application (this can be explored and approximated, under general condition, with the use of suitable computational modules ... that even Jgrass-NewAGE does not posses).

Finally and besides, establishing what is wet an what is dry implies to know how much the initial soil moisture is. So, even if we cut ET with soil moisture we cannot do it properly, if we do not do a guess about soil moisture initial condition.

So what can we do to estimate alpha of PT and having a guess of the initial soil moisture ?

The answer in one of the next posts.

P.S. - A not up-to-date series of slides on ET can be found on Slideshare here.

Monday, December 1, 2014

Luca Brocca interview on Research Gate

Luca Brocca recently was very much interviewed for one of his achievements about the use of remote sensing in hydrology. He had this smart idea of obtaining rainfall from soil-moisture data. His SM2RAIN is a simple algorithm for estimating rainfall from soil moisture data that you can find in his web page together with  other interesting stuff:

The paper that generate a big wave was:

Soil as a natural rain gauge: Estimating global rainfall from satellite soil moisture data,  available here. He also had the honour of a Nature Research Highlight mention. All of this deserve mention by itself. However, he was so kind to mention me in this recent Research Gate Interview. Thank you Luca !

Sunday, November 30, 2014

H2O - The Java way to Machine Learning

I discover H2O thanks to the R community where I found a post about the connection of this tool with R. Besides the interest for Machine Learning and Statistics (Data Science) which has been increasing during this year (yes, I did not do anything with it: but I need to learn before, and I take years to do it ;-( ), I was intrigued by the fact that it is implemented in Java, is said to be fast, has connection to R, Scala and Python, and has among the advisors Joshua Bloch, so, I guess, it should be REALLY good Java from which to learn.  Obviously I did add it to my previous overviews of Java  and R tools for hydrologists.

If they are really good as they promised, there are several things that we can copy from them: the connection to R, their class to read data, their strategies for doing math.

The tool is open source, information and download can be found at H2O site

Thursday, November 27, 2014

Ning Lu lectures on hillslope processes and (especially) stability, at the Summer School on Landslides

In 2013 University of Calabria organised a very interesting School on Landslide triggering (many thanks to Lino Versace, Giovanna Capparelli and Giuseppe Formetta).  I actually gave a hand to organised it, and  I also gave a lecture on Richards equation.  Waiting for the official post of the lectures at the school site (after which, I will remove my videos), I cannot wait anymore to have on-line the lectures by Ning Lu. He gave four talks taken out of his beautiful book, Hillslope Hydrology and Stability, written with Jonathan Godt, new coordinator of the USGS landslide hazards program, and former co-advisor of my Ph.D. student Silvia Simoni (her thesis here).  A must-watch for any guy in the field !

First talk: A brief conceptual history of soil hydrology and soil mechanics (from Chapter 6 of his book)

Third talk, part II: Hydro-mechanical properties of hillslopes (Chapter 8 of the book)

Fourth talk, part I: Failure surfaces  (Chapter 9 the book)

Fourth talk, part II: Field based stability analysis (Chapter 10 of his book)