the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Solving crustal heat transfer for thermochronology using physicsinformed neural networks
Abstract. We present a deep learning approach based on the physicsinformed neural networks (PINNs) for estimating thermal evolution of the crust during tectonic uplift with a changing landscape. The approach approximates the temperature field of the crust with a deep neural network, which is trained by optimizing the heat advectiondiffusion equation under boundary conditions such as initial and final thermal structure, topographic history, and surface and basal temperatures. From the trained neural network of temperature field and the prescribed velocity field, one can predict the temperature history of a given rock particle that can be used to compute the cooling ages of thermochronology. For the inverse problem, the forward model can be combined with a global optimization algorithm that minimizes the misfit between predicted and observed thermochronological data, in order to constrain unknown parameters in the uplift history or boundary conditions. We demonstrate the approach with solutions of one and threedimensional forward and inverse models of the crustal thermal evolution, which are consistent with results of the finiteelement method. The threedimensional model simulates the postorogenic topographic decay of the Dabie Shan, China, with constraints from fissiontrack and (UTh)/He ages.
 Preprint
(5054 KB)  Metadata XML
 BibTeX
 EndNote
Status: closed

RC1: 'Comment on gchron202324', Kendra Murray, 23 Nov 2023
General Comments:
This manuscript presents a new approach for solving the crustal heat transfer problem in 1D and 3D using a recently developed deeplearning approach called physicsinformed neural networks (PINNs). A PINN method can solve PDEs like the advectiondiffusion equation with complex boundary conditions, especially as inverse problems, and it is therefore an attractive tool for calculating the thermal structure of the crust given changing surface elevation (1D) or topography (3D)—a common task in the interpretation of cooling ages. This manuscript presents forward and inverse solutions with a simple synthetic scenario in 1D, demonstrating that PINNs can predict crustal thermal profiles and timetemperature paths (forward) similarly to a more traditional numerical finiteelement method and resolve the “true” exhumation histories, temperature histories, and synthetic cooling ages well (inverse). These 1D solutions are useful for demonstrating some of the model design choices that control optimization, search algorithms, etc. Then, this contribution moves into 3D, using an example of the CretaceousRecent landscape evolution of the Dabie Shan (the site of previous thermochronology studies and a published dataset). Forward and inverse models include changes to both topographic relief and rock uplift (and thus exhumation) rates over time. Inversion results are compared to PECUBE (an existing 3D finiteelement method that is most similar to what PINNs can in principle do) and published thermochronologic data from the region. Overall, this manuscript demonstrates that PINNs are capable of being trained to solve relatively simple crustal heat transfer problems, and they have potential for tackling more complex thermokinematic scenarios with more statistical rigor than what is presented in this contribution.
This manuscript provides an unprecedented perspective on how deeplearning approaches might be applied in thermochronology, and this is important because I think the next generation of computational tools we use to interpret cooling ages could benefit from such advances. By applying PINNs to the crustal thermal field problem, even with simple scenarios that our current tools are very capable of solving, this manuscript represents an important step in introducing deeplearning approaches to those of us interested in interrogating the thermal history of rocks using thermochronology. As such, with some revision for clarity (most of my suggestions are primarily provided as comments in the attached PDF), I think this contribution is a good fit for this journal. I do note that my expertise lies in thermal history and thermokinematic modeling of thermochron data, and not in neural networks, so I cannot really rigorously review the details of the PINN methods, functions, optimization, and learning rates described here. Instead, I focus my comments on the packaging of the method and how to make it accessible to a thermochron audience who might be considering using such tools.
Several specific comments that summarize my linebyline feedback in the PDF:
1. This manuscript would be stronger if it clearly articulated why deeplearning approaches will (eventually?) be an improvement over current numerical methods. On the face of them, PINNs seem like huge black boxes — with “multiple hidden layers with trainable parameters and nonlinear activation functions” (lines 967). Make a more clear case for why pursuing deeplearning approaches like PINNs is worthwhile. What will we get? Why should I spend time learning about this? It seems clear from this work that these methods are still in the early stages of development, and have not yet superseded currently available methods. Or, if the authors think the PINN approach presented here is already an improvement over current methods in some way (computational time? accuracy?), they do not make a clear case for this. Either way, a more explicit argument for the utility of these methods would improve the impact of this paper.
I will also add that the synthetic 1D scenario and model design is set up such that the inversion is a pretty simple problem to converge on the “true” answer for, which limits the impact of the example a bit. A suite of 6 perfect thermochronologic ages that span a 400˚C temperature range produced by a simple twostage monotonic cooling history in 1D is a very easy synthetic dataset to fit because it is distinctive — there is a narrow range of tT histories that can produce such a data set. This is especially true if, as is the case here, the inverse model design is narrow and dialed to only explore very similar/relevant histories (1D, monotonic, only exploring for one change in exhumation rate, only exploring 50 Ma of time, etc). Essentially, it’s not a hard problem to solve.
The synthetic 1D scenario presented here is an important starting demonstration, but many of the tools introduced in the beginning of this paper would perform similarly—no PINN black box required. I think a more interesting and compelling demonstration of the potential of this tool would be with a 1D (singlesample) synthetic dataset that is far less unique and more similar to a real dataset (fewer data) paired with a model design that explores whether these data could have been produced by a more complex or less constrained thermal history (i.e., starting earlier, non monotonic, multiple changes in exhumation rate possible). How well does this approach retrieve the “true” answer in this case? This is the realm of real thermochronologic datasets, and gets to the heart of the fundamental nonuniqueness of cooling ages, which is at the core of why thermochronologic studies rely on numerical tools for data interpretation.
The manuscript as it is already represents a complete contribution in my view, but I wanted to point out this additional opportunity.
2. Be more explicit about what is meant by “uplift” through out the paper (i.e., England and Molnar, 1990). The authors know that rock uplift only results in rock cooling if it is accompanied by exhumation, but that is not clear in the text as written. In the simple 1D scenarios, rock uplift = exhumation (both in rate and magnitude) because the surface elevation is not changing. But even then, using these terms interchangeably is not useful because there are many 1D scenarios where that would not be true. In the 3D example, the imposed rock uplift rate changes independently of the topographic evolution, so it is effectively decoupled from exhumation in space and time. Therefore, it is even more critical to be clear.
3. The model design and results are difficult to follow in places because it is hard to keep track of “model time” vs. “geologic time”, which are always opposites in such models. I suggest the authors can avoid some of this confusion by using “Myr” (millions of years) when referring to model time and timescales and “Ma” (millions of years ago) when referring to geological time and thermochronologic ages, in both the main text and figures.
4. The Dabie Shan example would benefit from some additional text describing the previous work (data, what is known about the orogenic history and timing) in more detail, so the model design decisions made have more geologic context.
5. The paper would benefit from a short conclusions section that summarizes the main findings and looks ahead more broadly to future applications of deeplearning approaches in thermochronology.

AC2: 'Reply on RC1', Ruohong Jiao, 23 Jan 2024
1 In the current version, PINNs are only used to approximate the thermal fields in the forward model. For the inversion, i.e., to estimate the possible thermal history from thermochronological ages, we use a global search method instead of PINNs. As the reviewer pointed out, in a lot of cases for a set of cooling ages, the possible timetemperature history is not unique. In this case, our search method won’t be able to “retrieve the ‘true’ answer”, nor any other search methods can do. As the main purpose of this work is to introduce the PINN method, in the revision we can compare performance of inversions in which the thermal histories are solved using the PINNs vs. numerical methods. We could test with a more complex “true” model, but without including fissiontrack length or singlegrain He data it is unlikely for the inversion to resolve a nonmonotonic history.
Currently, the biggest advantage of PINNs over traditional numerical methods is mesh free. For application of finiteelement and finitedifference methods, one has to create grids. This can be cumbersome for complicate and timedependent boundary conditions which are often in geological problems. In the current paper, we focus on demonstrating the feasibility of the PINNs method, and thus verify the results by comparing the results with that of numerical methods. Therefore, we did not design too complicate problems but such problems will be target in future developments of the method.
PINNs also have other potential advantages that are not realised in the current version yet, such as quickly solving inverse problems and convenient integration of multisource data of different scales.
2 Throughout the paper, “uplift” is “rock uplift” unless otherwise specified. In the revision, we will be careful and more specific by using “rock uplift” instead of “uplift”.
3 Good idea. We will use “Myr” and “Ma” for “model time” and “geological time”, respectively.
4 We will add more background and introduction to the Dabie Shan study.
5 We will add a Conclusion section.

Linebyline responses
L14 Will be changed to “how thermal structure of the crust has changed in time and space must be understood”.
L18 Will add references He and Tartakovsky (2021), RashtBehesht et al. (2022).
L20 The method is easier to implement than numerical methods, and therefore has the potential to apply more complex boundary conditions. The introduction will be expanded to explain this point.
L21 Rock uplift. The text will be modified.
L28 Agreed. “Quantifying” is a better term.
L30 Text will be changed to “Collected on a vertical profile”
L31 “adopts” will be changed to “based on”. Next sentence will be changed to explicitly say “the method does not require calculation of Tc”. Limitation of AER outside “steadystate” is mentioned in L37. The difficulty of collecting vertical profile will be added to the end of paragraph.
L40 Even when modeling data on a vertical profile, QTQt (at least the recent version) does not solve heat transfer; instead, it applies a linear interpolation to get Tt histories of samples and the calculate their ages. My focus here is not the Tt history modelling part of QTQt or HeFTy. Rather, some studies have used modelled Tt histories, which can be retrieved from the two apps or any other approaches, to estimate the geothermal gradient changes over time. I will rewrite this paragraph to avoid confusion.
L42 QTQt has a forward modelling function to compute thermochron ages from Tt history. Hefty also has the same function but perhaps not calls it “forward”.
L64 Again, QTQt and HeFTy are not considered apps or methods to calculate exhumation rates here, but the manipulation of the modelled Tt histories (as the reviewer stated, “it is up to the users to interpret these histories” ) is a method. Also, “the methods mentioned above” are only methods in this section 2.2. Text will be modified to clarify.
L65 Text will be changed to “a thermochronometer’s cooling age”.
L90 Numerical method can be difficult to implement. Pecube has a function to calculate velocity fields associated with a single fault, or a set of faults with the same strike. A method which is convenient to implement more complex boundary conditions will be useful for solving exhumation in more complicate settings, although they have not been tested in this work yet.
L100 “demonstrate the setup of the neural networks”
L104 The value of the method will be added.
L127 Model time will be changed to use the unit Myr.
L129 Will use “exhumation rate”.
L141 It is for demonstration purpose and a personal preference. Other machine learning libraries, e.g., PyTorch, will work too.
L150 I can add a sentence to explain the FEM.
L166 Here is a general statement. More detailed setup of the inverse problem will be described below, as the reviewer noted.
L177 The sentence will be modified.
L209 Yes. In the revision, I could add the result from a FEM + MCMC method as a comparison.
L212 Typo will be corrected.
L214 A summary will be added.
L219 It should be the last phase of orogeny and postorogenic exhumation. Sentence will be modified with more context added above.
L221 Yes. It should be ‘y’. The range of z includes potential maximum elevation of the Dabie Shan during mountain building, prior to postorogenic decay.
L235 This should include the last phase of orogeny. Text in the model setup will be modified to reflect this.
L244 Yes, it is “rock uplift”. Exhumation is not calculated here, but it should be the summation of rock uplift and topographic decay.
L247 PINN (physicsinformed neural network) is a type of DNN (deep neural network) with the physical laws embedded (as partial differential equations).
L265 Here should be the model time, i.e., 150 Myr, or 0 Ma. I will clarify this.
L269L271 These norms reflect the difference in ages calculated using PINN and FEM. The l^2 norm sums misfits from all data, so its value depends on the number of data points. I will provide equations for calculating l^2 l^inf for easier understanding of the results.
L275 Descriptions of the values will be added.
L299 There is no need to put an equation here as I did not model it. Here I mean “the rock uplift in our model resulted from tectonic uplift and isostatic rebound.” The text will be modified for clarification.
L309 “assimilation” is a typo; it should be “simulation”.
L311 It should be “empirical data”.
L312 “applications for geological problems with real data.”
L330 A conclusion section will be added.
Figure 14 Good idea. I will add sub figures to show the difference.
Citation: https://doi.org/10.5194/gchron202324AC2

AC2: 'Reply on RC1', Ruohong Jiao, 23 Jan 2024

RC2: 'Comment on gchron202324', David Whipp, 18 Dec 2023
Summary
Jiao et al. presents an approach to determining the cooling and exhumation history of bedrock thermochronometer samples using physicsinformed neural networks. The method presented here has not, as far as I am aware, been applied to thermochronology previously and is a flexible and modern alternative to the use of finiteelement or finitedifference software for solving the heat transfer equation, calculating thermal histories, and predicting thermochronometer ages. The authors also demonstrate the potential use in forward models and for data inversion in one or multiple spatial dimensions in two example cases. Overall, the manuscript is a good fit for Geochronology, the method certainly seems appealing, and a nice case is made for its potential application. However, there are parts of the manuscript that would benefit from being expanded or otherwise edited to ensure readers have a good sense of the pros, cons, and potential for this method, and a handful of other considerations that could make this a stronger manuscript. In the comments below I note some of the general issues that should be considered followed by more detailed comments about the manuscript text and figures.
Main comments
 One of the main things I feel needs more attention in the manuscript is the case for why someone should consider using this method for interpreting their data. As detailed below, there are many interesting results that are presented, but the case for using this approach compared to existing methods is not really made/emphasized. As a new method being pitched, this seems like a key point to ensure readers are aware of how they might benefit.
 It would also be good, given that I would imagine many readers are not that familiar with neural networks, to provide a bit more background on topics like what neural networks are and how they differ in terms of solving equations from numerical methods like the finiteelement or finitedifference methods. There are details for some things, but it also seems to be that familiarity with some neural network topics is assumed, which might not be true for many interested readers. A bit more background could significantly improve the ability of a broader group of readers to benefit from this work.
 This is a bit tricky to handle with this type of paper, but topics typical for the methods section are mixed with the results in sections 4 and 5. This might make it hard for readers to find certain technical details, which would be unfortunate given this is a new method that readers might want to test. I would suggest considering whether it is possible to more cleanly split things between the results and methods topics in sections 4 and 5.
 The discussion of differences between the FE and PINN solutions for the heat transfer equations is helpful in illustrating that the PINNs do reproduce the expected temperatures fairly well, but it might also be nice to have some basis for comparison. For example, it would be nice to see also some information about how much a typical finiteelement solution differs from analytical solutions for heat transfer. In addition, I would suggest considering reporting percent differences between the two temperature solutions rather than absolute temperature differences, as the percent difference would not be linked to the temperature bounds the boundary conditions impose, for example. The same applies to the predicted ages. I have tried to note places where this could apply in the specific remarks below.
 Finally, the manuscript is generally well written and clear, but there are some large contrasts in terms of how detailed different sections of the manuscript are. For example, the introduction section is shorter than the text describing the reason for using the differential evolution algorithm in the inverse problem description for the 1D case. As a result, some sections feel underdeveloped and lacking compared to sections that are rich with detail and information. I have tried to note sections where further development would help in the specific remarks below.
Specific remarks (L = line number)
L111: Somewhere in the abstract it would be good to highlight the advantages of this approach over some of the alternatives that are currently used. It might also be helpful to note why the Dabie Shan case study is used (briefly).
L1224: The introduction as a whole is quite short and doesn't really make a case for why this method would appeal to readers. For instance, it would be good to include something about some drawbacks of the current methods or things that motivate the use of PINNs. Essentially, similar to the abstract, it seems a case should be made for why this method should be used. The section that follows goes into more detail about existing approaches for finding thermal histories or linking ages to rates of exhumation, but before moving to that section it would be good to know what we might gain from the new approach.
L3132: I am not sure this statement is correct. Although the ageelevation method for calculating exhumation rates does make some assumptions about the geometry of the effective closure isotherm in the subsurface (see Figure 2 of Ehlers, 2005, Reviews in Mineralogy and Geochemistry, for example), there is no need to calculate an effective closure temperature to use an ageelevation profile to estimate an exhumation rate. You simply plot the sample age versus its elevation and the slope of a bestfit line is the exhumation rate estimate. I suggest not mentioning Dodson here, or at least clarifying that there is no need to actually consider the closure temperature to use this approach.
L3537: This is true, but it is also worth noting that there are other factors that can make this approach not work (such as those noted in the Ehlers paper mentioned above). Topographic and exhumational steady state can exist and still cause problems with the estimated exhumation rate due to topography affecting the effective closure temperature isotherm geometry.
L4041: I suppose here it might be good to introduce the thermal history modelling from HeFTy or QTQt first since many readers might know those models and then bring in the possible links to differences in sample depth or elevation. It seems quite handy to exploit the elevation to be able to calculate exhumation rates from codes like HeFTy, but for simplicity it might be good to present their general use first.
L52: This subsection might be more simply titled "Calculating crustal temperatures"
L6472: Although we're in the process of writing a first article using the software, you could mention the Tc1D software (https://zenodo.org/doi/10.5281/zenodo.7124271) as another 1D code for calculating exhumation rates from thermochronometer data. The code solves the 1D heat transfer equation using finite differences and calculates (UTh)/He and fissiontrack ages for apatite and zircon. Several different exhumation history models can be used to simulate timevarying exhumation rates and cooling scenarios.
L91: As noted earlier in the introduction and abstract, it seems important that somewhere in this section you point out some of the deficiencies of the current software used for interpreting thermochronology data to make an appealing case for the use of PINNs. Without this, it feels somewhat like the method is simply a neat computational development.
L9397: Given the range of potential readers (as noted in the general comments), it might be helpful to add a few sentences with general background about neural networks and how they differ from the numerical methods often used for similar heat transfer problems.
L117: I would move the text from lines 126130 here to first describe the general scenario this first forward model explores in words before describing the boundary conditions, exhumation rate model, etc.
L131135: It might be again beneficial here to explain a bit more about how the loss function components are calculated and why the weighting parameter has its assigned value. For instance, what is the basis for comparison in finding the residuals for Lf?
L148149: Figure 3 shows some "spikes" that appear in Lb after about 5000 iterations. While L appears to nicely decrease, is there any meaning to the "spikes" in Lb and their apparent growth with time? Should this be discussed somewhere?
L156157: The errors in degrees are helpful here, but perhaps it would also be good to note the percent error, at least for the infinity norm.
L163164: Similar comment to above, that it might be nice to have a percent difference in age here.
L177189: This is a nice paragraph detailing and important choice in the design of the inverse model approach. However, the level of detail here is much greater than other sections of the text (I found this to be a good thing), so it makes those other sections appear to be lacking detail. In addition, this is an example of a topic that would seemingly belong in the methods section rather than in the results (although these are mixed in section 4).
L208: Again, perhaps it is worth considering using percent difference here.
L209210: This sentence wording was somewhat awkward. Perhaps this could be rephrased.
L212213: It would be worthwhile to note that this a a considerably more complex and demanding example compared to the first case.
L235237: It would be helpful to have the text description of the topographic evolution moved to be prior to equation 17, for ease of reading. Also, I suggest using "Myr" to refer to model time (duration) and "Ma" to refer to millions of years ago (i.e., geochronological time). This is the first place I noted this issue in the text, but the "Myr" and "Ma" idea could be applied throughout the text and figures.
L260: The "i.e." is not needed here. I suggest "...using the FEM code Pecube (Braun, 2003)."
L264, 270271: Again, please consider percent difference as an option for the infinity norm here.
L273: The first sentence here is quite vague and should be made more specific to the case presented in this subsection.
L276277: Perhaps not here, but it would be worthwhile to know how much more demanding the inversion would be with all 5 free parameters. Could this be revisited in the discussion somewhere?
L286288: As the total number of iterations is reported in the previous paragraph, could the total number of iterations be listed here as well?
L292294: It seems that the convergence behavior would be another interesting point for further exploration in the discussion.
L296301: This seems like a paragraph that belongs in the discussion section, perhaps revisiting the results of the examples that have been presented and how/why the results differ from the FEM solutions or estimates from other inversions. As the discussion section is currently fairly short, perhaps this could be moved and expanded?
L304312: This section is probably the one I would seek out as a reader and it is currently quite brief. It would be nice to expand this and add some structure (i.e., multiple paragraphs) to discuss each of the potential benefits in more detail. For instance, I would be interested to know more about the computational efficiency of this approach compared to the alternatives, and this is only briefly mentioned in the current text. My sense is that the 3D PINNs can be quite fast, so this would be important to highlight if you want to interest readers in applying this method.
L315: "CPU platform" seems a bit odd to me. Perhaps there is another term to use, such as "traditional CPUbased systems"?
Figure 2: Perhaps the units for the lower xaxes could be "Myr"?
Figure 4: My experience has been that it might be easier to see how the error varies if these plots were made as percent error, rather than temperature on the x axis. Perhaps this could be considered here.
Figure 5: Perhaps the loglog plot in panel b makes things look nicer, but it makes it a bit harder to visualize the error in predicted age. Perhaps a percent error could be considered here as well.
Figure 6: It is quite hard to see the blue curves in general, and in panel a, it seems the value of the total loss is still decreasing for the green model. Is there any reason to think the loss would not decrease more for that case?
Figure 7: The subplot labels are hard to see in some plots. Perhaps the background could be solid white?
Figure 10: The lower xaxes could again use "Myr" for their unit.
Figure 13: Could these be plots using percent difference?
Figure 14: It might be easier to compare the models if plotted as percent difference here (again). It would also require only two panels. And I would encourage using a colormap that is safe for readers with color blindness (e.g., https://www.fabiocrameri.ch/colourmaps/).
Figure 17: Again, "Myr" could be used here.
Technical comments
L139: This line should not be indented.
L212: "Figure 9" misspelled in citation.
L296: "...with Braun and Robert..."
Citation: https://doi.org/10.5194/gchron202324RC2 
AC1: 'Reply on RC2', Ruohong Jiao, 23 Jan 2024
1 We will add more text to explain the benefit of the method. Reviewer 1 also pointed this out.
2 We will add more detailed introduction of neural networks.
3 We will separate the method and result sections where possible.
4 We will add percentages in the comparisons of temperature solutions between the PINN and FE.
5 We will elaborate the text where necessary, as noted by the reviewer.

Linebyline responses
L111 Highlight of PINNs will be added. Dabie Shan has a well studied postorogenic history, simple tectonic setting, and has been used as a site for applying or testing different modelling techniques. Abstract will be modified.
L1224 Good point. This will be a focus for revising the manuscript. Reviewer 1 also raised the same point.
L3132 The statement will be rewritten to avoid confusion.
L3537 Other factors that affect the reliability of AER will be added. Thanks for the reference.
L4041 Agreed. The current paragraph has caused some confusion, and I agreed that it is worth to introduce the Tt history modelling techniques. The paragraph will be rewritten for clarification.
L52 Title will be changed to “calculating crustal temperatures”.
L6472 I will check Tc1D and add it here.
L91 Agreed. I will emphasise this in the revision.
L9397 A short introduction of neural networks will be added.
L117 Good idea. I will do that.
L131135 I will improve the writing here. L_f is calculated on all collocation points with the governing PDEs, equation 1 in this case.
L148149 The spikes indicate that the learning rate was set too high. In the revision, I could set the a stepwise learning rate to remove or reduce the occurrence of such spikes.
L156157 I will add the error in percentage forms.
L163164 I will add the error in percentage forms.
L177189 Thanks. The methods in other sections will be elaborated. I will separate the method and result sections in each model experiment.
L208 Yes. I will add that.
L209 Yes. Reviewer 1 also thinks it is confusing here. I want to say that the solutions (exhumation histories) to the synthetic data may not be unique for the slow exhumation period. I will verify this by adding result using numerical methods, and compare it to that from PINNs+ DE.
L213 I will do that, emphasising the challenges of 3D models.
L235237 Good suggestion. I will rearrange the text and use “Myr” to indicate model time.
L260 I will remove it.
L264, 270271 Percentage form of norms will be added.
L273 It will be changed to “To test the capability of PINNs in solving an inverse problem, we try to solve the posorogenic uplift history of Dabie Shan using published lowtemperature thermochronological ages.”
L276277 I will explore this in the inversion, but exploration of the 5D space could be time consuming with my current computing power.
L286288 Yes. I will add that. It’s 1,800.
L292294 I will investigate the behaviour of DE and add it in discussion.
L296301 I will investigate and discuss this misfits in more detail. One possible way is to rerun the Pecube modelling without the isostatic compensation component.
L304312 I will elaborate the advantage or potentials of PINNs in the revision. Reviewer 1 also mentioned that this could be added to the instruction. I will carefully revise and make sure the rewriting will be in detail but not repetitive between introduction and discussion.
L315 Will change to “CPUbased system”
Figure 2 Yes. It will be changed.
Figure 4 Good idea. I will add panels to show the percentage.
Figure 5 I will add that.
Figure 6 The models will be run longer to get a more stable loss curve.
Figure 7 I will change the background colour to white.
Figure 10 Yes. It will be changed.
Figure 13 It is possible. I will add plots for better comparison of the results.
Figure 14 I will add panels showing the difference and change the colormap.

Technical comments
L139, L212, L296 Typos will be corrected.
Citation: https://doi.org/10.5194/gchron202324AC1
Status: closed

RC1: 'Comment on gchron202324', Kendra Murray, 23 Nov 2023
General Comments:
This manuscript presents a new approach for solving the crustal heat transfer problem in 1D and 3D using a recently developed deeplearning approach called physicsinformed neural networks (PINNs). A PINN method can solve PDEs like the advectiondiffusion equation with complex boundary conditions, especially as inverse problems, and it is therefore an attractive tool for calculating the thermal structure of the crust given changing surface elevation (1D) or topography (3D)—a common task in the interpretation of cooling ages. This manuscript presents forward and inverse solutions with a simple synthetic scenario in 1D, demonstrating that PINNs can predict crustal thermal profiles and timetemperature paths (forward) similarly to a more traditional numerical finiteelement method and resolve the “true” exhumation histories, temperature histories, and synthetic cooling ages well (inverse). These 1D solutions are useful for demonstrating some of the model design choices that control optimization, search algorithms, etc. Then, this contribution moves into 3D, using an example of the CretaceousRecent landscape evolution of the Dabie Shan (the site of previous thermochronology studies and a published dataset). Forward and inverse models include changes to both topographic relief and rock uplift (and thus exhumation) rates over time. Inversion results are compared to PECUBE (an existing 3D finiteelement method that is most similar to what PINNs can in principle do) and published thermochronologic data from the region. Overall, this manuscript demonstrates that PINNs are capable of being trained to solve relatively simple crustal heat transfer problems, and they have potential for tackling more complex thermokinematic scenarios with more statistical rigor than what is presented in this contribution.
This manuscript provides an unprecedented perspective on how deeplearning approaches might be applied in thermochronology, and this is important because I think the next generation of computational tools we use to interpret cooling ages could benefit from such advances. By applying PINNs to the crustal thermal field problem, even with simple scenarios that our current tools are very capable of solving, this manuscript represents an important step in introducing deeplearning approaches to those of us interested in interrogating the thermal history of rocks using thermochronology. As such, with some revision for clarity (most of my suggestions are primarily provided as comments in the attached PDF), I think this contribution is a good fit for this journal. I do note that my expertise lies in thermal history and thermokinematic modeling of thermochron data, and not in neural networks, so I cannot really rigorously review the details of the PINN methods, functions, optimization, and learning rates described here. Instead, I focus my comments on the packaging of the method and how to make it accessible to a thermochron audience who might be considering using such tools.
Several specific comments that summarize my linebyline feedback in the PDF:
1. This manuscript would be stronger if it clearly articulated why deeplearning approaches will (eventually?) be an improvement over current numerical methods. On the face of them, PINNs seem like huge black boxes — with “multiple hidden layers with trainable parameters and nonlinear activation functions” (lines 967). Make a more clear case for why pursuing deeplearning approaches like PINNs is worthwhile. What will we get? Why should I spend time learning about this? It seems clear from this work that these methods are still in the early stages of development, and have not yet superseded currently available methods. Or, if the authors think the PINN approach presented here is already an improvement over current methods in some way (computational time? accuracy?), they do not make a clear case for this. Either way, a more explicit argument for the utility of these methods would improve the impact of this paper.
I will also add that the synthetic 1D scenario and model design is set up such that the inversion is a pretty simple problem to converge on the “true” answer for, which limits the impact of the example a bit. A suite of 6 perfect thermochronologic ages that span a 400˚C temperature range produced by a simple twostage monotonic cooling history in 1D is a very easy synthetic dataset to fit because it is distinctive — there is a narrow range of tT histories that can produce such a data set. This is especially true if, as is the case here, the inverse model design is narrow and dialed to only explore very similar/relevant histories (1D, monotonic, only exploring for one change in exhumation rate, only exploring 50 Ma of time, etc). Essentially, it’s not a hard problem to solve.
The synthetic 1D scenario presented here is an important starting demonstration, but many of the tools introduced in the beginning of this paper would perform similarly—no PINN black box required. I think a more interesting and compelling demonstration of the potential of this tool would be with a 1D (singlesample) synthetic dataset that is far less unique and more similar to a real dataset (fewer data) paired with a model design that explores whether these data could have been produced by a more complex or less constrained thermal history (i.e., starting earlier, non monotonic, multiple changes in exhumation rate possible). How well does this approach retrieve the “true” answer in this case? This is the realm of real thermochronologic datasets, and gets to the heart of the fundamental nonuniqueness of cooling ages, which is at the core of why thermochronologic studies rely on numerical tools for data interpretation.
The manuscript as it is already represents a complete contribution in my view, but I wanted to point out this additional opportunity.
2. Be more explicit about what is meant by “uplift” through out the paper (i.e., England and Molnar, 1990). The authors know that rock uplift only results in rock cooling if it is accompanied by exhumation, but that is not clear in the text as written. In the simple 1D scenarios, rock uplift = exhumation (both in rate and magnitude) because the surface elevation is not changing. But even then, using these terms interchangeably is not useful because there are many 1D scenarios where that would not be true. In the 3D example, the imposed rock uplift rate changes independently of the topographic evolution, so it is effectively decoupled from exhumation in space and time. Therefore, it is even more critical to be clear.
3. The model design and results are difficult to follow in places because it is hard to keep track of “model time” vs. “geologic time”, which are always opposites in such models. I suggest the authors can avoid some of this confusion by using “Myr” (millions of years) when referring to model time and timescales and “Ma” (millions of years ago) when referring to geological time and thermochronologic ages, in both the main text and figures.
4. The Dabie Shan example would benefit from some additional text describing the previous work (data, what is known about the orogenic history and timing) in more detail, so the model design decisions made have more geologic context.
5. The paper would benefit from a short conclusions section that summarizes the main findings and looks ahead more broadly to future applications of deeplearning approaches in thermochronology.

AC2: 'Reply on RC1', Ruohong Jiao, 23 Jan 2024
1 In the current version, PINNs are only used to approximate the thermal fields in the forward model. For the inversion, i.e., to estimate the possible thermal history from thermochronological ages, we use a global search method instead of PINNs. As the reviewer pointed out, in a lot of cases for a set of cooling ages, the possible timetemperature history is not unique. In this case, our search method won’t be able to “retrieve the ‘true’ answer”, nor any other search methods can do. As the main purpose of this work is to introduce the PINN method, in the revision we can compare performance of inversions in which the thermal histories are solved using the PINNs vs. numerical methods. We could test with a more complex “true” model, but without including fissiontrack length or singlegrain He data it is unlikely for the inversion to resolve a nonmonotonic history.
Currently, the biggest advantage of PINNs over traditional numerical methods is mesh free. For application of finiteelement and finitedifference methods, one has to create grids. This can be cumbersome for complicate and timedependent boundary conditions which are often in geological problems. In the current paper, we focus on demonstrating the feasibility of the PINNs method, and thus verify the results by comparing the results with that of numerical methods. Therefore, we did not design too complicate problems but such problems will be target in future developments of the method.
PINNs also have other potential advantages that are not realised in the current version yet, such as quickly solving inverse problems and convenient integration of multisource data of different scales.
2 Throughout the paper, “uplift” is “rock uplift” unless otherwise specified. In the revision, we will be careful and more specific by using “rock uplift” instead of “uplift”.
3 Good idea. We will use “Myr” and “Ma” for “model time” and “geological time”, respectively.
4 We will add more background and introduction to the Dabie Shan study.
5 We will add a Conclusion section.

Linebyline responses
L14 Will be changed to “how thermal structure of the crust has changed in time and space must be understood”.
L18 Will add references He and Tartakovsky (2021), RashtBehesht et al. (2022).
L20 The method is easier to implement than numerical methods, and therefore has the potential to apply more complex boundary conditions. The introduction will be expanded to explain this point.
L21 Rock uplift. The text will be modified.
L28 Agreed. “Quantifying” is a better term.
L30 Text will be changed to “Collected on a vertical profile”
L31 “adopts” will be changed to “based on”. Next sentence will be changed to explicitly say “the method does not require calculation of Tc”. Limitation of AER outside “steadystate” is mentioned in L37. The difficulty of collecting vertical profile will be added to the end of paragraph.
L40 Even when modeling data on a vertical profile, QTQt (at least the recent version) does not solve heat transfer; instead, it applies a linear interpolation to get Tt histories of samples and the calculate their ages. My focus here is not the Tt history modelling part of QTQt or HeFTy. Rather, some studies have used modelled Tt histories, which can be retrieved from the two apps or any other approaches, to estimate the geothermal gradient changes over time. I will rewrite this paragraph to avoid confusion.
L42 QTQt has a forward modelling function to compute thermochron ages from Tt history. Hefty also has the same function but perhaps not calls it “forward”.
L64 Again, QTQt and HeFTy are not considered apps or methods to calculate exhumation rates here, but the manipulation of the modelled Tt histories (as the reviewer stated, “it is up to the users to interpret these histories” ) is a method. Also, “the methods mentioned above” are only methods in this section 2.2. Text will be modified to clarify.
L65 Text will be changed to “a thermochronometer’s cooling age”.
L90 Numerical method can be difficult to implement. Pecube has a function to calculate velocity fields associated with a single fault, or a set of faults with the same strike. A method which is convenient to implement more complex boundary conditions will be useful for solving exhumation in more complicate settings, although they have not been tested in this work yet.
L100 “demonstrate the setup of the neural networks”
L104 The value of the method will be added.
L127 Model time will be changed to use the unit Myr.
L129 Will use “exhumation rate”.
L141 It is for demonstration purpose and a personal preference. Other machine learning libraries, e.g., PyTorch, will work too.
L150 I can add a sentence to explain the FEM.
L166 Here is a general statement. More detailed setup of the inverse problem will be described below, as the reviewer noted.
L177 The sentence will be modified.
L209 Yes. In the revision, I could add the result from a FEM + MCMC method as a comparison.
L212 Typo will be corrected.
L214 A summary will be added.
L219 It should be the last phase of orogeny and postorogenic exhumation. Sentence will be modified with more context added above.
L221 Yes. It should be ‘y’. The range of z includes potential maximum elevation of the Dabie Shan during mountain building, prior to postorogenic decay.
L235 This should include the last phase of orogeny. Text in the model setup will be modified to reflect this.
L244 Yes, it is “rock uplift”. Exhumation is not calculated here, but it should be the summation of rock uplift and topographic decay.
L247 PINN (physicsinformed neural network) is a type of DNN (deep neural network) with the physical laws embedded (as partial differential equations).
L265 Here should be the model time, i.e., 150 Myr, or 0 Ma. I will clarify this.
L269L271 These norms reflect the difference in ages calculated using PINN and FEM. The l^2 norm sums misfits from all data, so its value depends on the number of data points. I will provide equations for calculating l^2 l^inf for easier understanding of the results.
L275 Descriptions of the values will be added.
L299 There is no need to put an equation here as I did not model it. Here I mean “the rock uplift in our model resulted from tectonic uplift and isostatic rebound.” The text will be modified for clarification.
L309 “assimilation” is a typo; it should be “simulation”.
L311 It should be “empirical data”.
L312 “applications for geological problems with real data.”
L330 A conclusion section will be added.
Figure 14 Good idea. I will add sub figures to show the difference.
Citation: https://doi.org/10.5194/gchron202324AC2

AC2: 'Reply on RC1', Ruohong Jiao, 23 Jan 2024

RC2: 'Comment on gchron202324', David Whipp, 18 Dec 2023
Summary
Jiao et al. presents an approach to determining the cooling and exhumation history of bedrock thermochronometer samples using physicsinformed neural networks. The method presented here has not, as far as I am aware, been applied to thermochronology previously and is a flexible and modern alternative to the use of finiteelement or finitedifference software for solving the heat transfer equation, calculating thermal histories, and predicting thermochronometer ages. The authors also demonstrate the potential use in forward models and for data inversion in one or multiple spatial dimensions in two example cases. Overall, the manuscript is a good fit for Geochronology, the method certainly seems appealing, and a nice case is made for its potential application. However, there are parts of the manuscript that would benefit from being expanded or otherwise edited to ensure readers have a good sense of the pros, cons, and potential for this method, and a handful of other considerations that could make this a stronger manuscript. In the comments below I note some of the general issues that should be considered followed by more detailed comments about the manuscript text and figures.
Main comments
 One of the main things I feel needs more attention in the manuscript is the case for why someone should consider using this method for interpreting their data. As detailed below, there are many interesting results that are presented, but the case for using this approach compared to existing methods is not really made/emphasized. As a new method being pitched, this seems like a key point to ensure readers are aware of how they might benefit.
 It would also be good, given that I would imagine many readers are not that familiar with neural networks, to provide a bit more background on topics like what neural networks are and how they differ in terms of solving equations from numerical methods like the finiteelement or finitedifference methods. There are details for some things, but it also seems to be that familiarity with some neural network topics is assumed, which might not be true for many interested readers. A bit more background could significantly improve the ability of a broader group of readers to benefit from this work.
 This is a bit tricky to handle with this type of paper, but topics typical for the methods section are mixed with the results in sections 4 and 5. This might make it hard for readers to find certain technical details, which would be unfortunate given this is a new method that readers might want to test. I would suggest considering whether it is possible to more cleanly split things between the results and methods topics in sections 4 and 5.
 The discussion of differences between the FE and PINN solutions for the heat transfer equations is helpful in illustrating that the PINNs do reproduce the expected temperatures fairly well, but it might also be nice to have some basis for comparison. For example, it would be nice to see also some information about how much a typical finiteelement solution differs from analytical solutions for heat transfer. In addition, I would suggest considering reporting percent differences between the two temperature solutions rather than absolute temperature differences, as the percent difference would not be linked to the temperature bounds the boundary conditions impose, for example. The same applies to the predicted ages. I have tried to note places where this could apply in the specific remarks below.
 Finally, the manuscript is generally well written and clear, but there are some large contrasts in terms of how detailed different sections of the manuscript are. For example, the introduction section is shorter than the text describing the reason for using the differential evolution algorithm in the inverse problem description for the 1D case. As a result, some sections feel underdeveloped and lacking compared to sections that are rich with detail and information. I have tried to note sections where further development would help in the specific remarks below.
Specific remarks (L = line number)
L111: Somewhere in the abstract it would be good to highlight the advantages of this approach over some of the alternatives that are currently used. It might also be helpful to note why the Dabie Shan case study is used (briefly).
L1224: The introduction as a whole is quite short and doesn't really make a case for why this method would appeal to readers. For instance, it would be good to include something about some drawbacks of the current methods or things that motivate the use of PINNs. Essentially, similar to the abstract, it seems a case should be made for why this method should be used. The section that follows goes into more detail about existing approaches for finding thermal histories or linking ages to rates of exhumation, but before moving to that section it would be good to know what we might gain from the new approach.
L3132: I am not sure this statement is correct. Although the ageelevation method for calculating exhumation rates does make some assumptions about the geometry of the effective closure isotherm in the subsurface (see Figure 2 of Ehlers, 2005, Reviews in Mineralogy and Geochemistry, for example), there is no need to calculate an effective closure temperature to use an ageelevation profile to estimate an exhumation rate. You simply plot the sample age versus its elevation and the slope of a bestfit line is the exhumation rate estimate. I suggest not mentioning Dodson here, or at least clarifying that there is no need to actually consider the closure temperature to use this approach.
L3537: This is true, but it is also worth noting that there are other factors that can make this approach not work (such as those noted in the Ehlers paper mentioned above). Topographic and exhumational steady state can exist and still cause problems with the estimated exhumation rate due to topography affecting the effective closure temperature isotherm geometry.
L4041: I suppose here it might be good to introduce the thermal history modelling from HeFTy or QTQt first since many readers might know those models and then bring in the possible links to differences in sample depth or elevation. It seems quite handy to exploit the elevation to be able to calculate exhumation rates from codes like HeFTy, but for simplicity it might be good to present their general use first.
L52: This subsection might be more simply titled "Calculating crustal temperatures"
L6472: Although we're in the process of writing a first article using the software, you could mention the Tc1D software (https://zenodo.org/doi/10.5281/zenodo.7124271) as another 1D code for calculating exhumation rates from thermochronometer data. The code solves the 1D heat transfer equation using finite differences and calculates (UTh)/He and fissiontrack ages for apatite and zircon. Several different exhumation history models can be used to simulate timevarying exhumation rates and cooling scenarios.
L91: As noted earlier in the introduction and abstract, it seems important that somewhere in this section you point out some of the deficiencies of the current software used for interpreting thermochronology data to make an appealing case for the use of PINNs. Without this, it feels somewhat like the method is simply a neat computational development.
L9397: Given the range of potential readers (as noted in the general comments), it might be helpful to add a few sentences with general background about neural networks and how they differ from the numerical methods often used for similar heat transfer problems.
L117: I would move the text from lines 126130 here to first describe the general scenario this first forward model explores in words before describing the boundary conditions, exhumation rate model, etc.
L131135: It might be again beneficial here to explain a bit more about how the loss function components are calculated and why the weighting parameter has its assigned value. For instance, what is the basis for comparison in finding the residuals for Lf?
L148149: Figure 3 shows some "spikes" that appear in Lb after about 5000 iterations. While L appears to nicely decrease, is there any meaning to the "spikes" in Lb and their apparent growth with time? Should this be discussed somewhere?
L156157: The errors in degrees are helpful here, but perhaps it would also be good to note the percent error, at least for the infinity norm.
L163164: Similar comment to above, that it might be nice to have a percent difference in age here.
L177189: This is a nice paragraph detailing and important choice in the design of the inverse model approach. However, the level of detail here is much greater than other sections of the text (I found this to be a good thing), so it makes those other sections appear to be lacking detail. In addition, this is an example of a topic that would seemingly belong in the methods section rather than in the results (although these are mixed in section 4).
L208: Again, perhaps it is worth considering using percent difference here.
L209210: This sentence wording was somewhat awkward. Perhaps this could be rephrased.
L212213: It would be worthwhile to note that this a a considerably more complex and demanding example compared to the first case.
L235237: It would be helpful to have the text description of the topographic evolution moved to be prior to equation 17, for ease of reading. Also, I suggest using "Myr" to refer to model time (duration) and "Ma" to refer to millions of years ago (i.e., geochronological time). This is the first place I noted this issue in the text, but the "Myr" and "Ma" idea could be applied throughout the text and figures.
L260: The "i.e." is not needed here. I suggest "...using the FEM code Pecube (Braun, 2003)."
L264, 270271: Again, please consider percent difference as an option for the infinity norm here.
L273: The first sentence here is quite vague and should be made more specific to the case presented in this subsection.
L276277: Perhaps not here, but it would be worthwhile to know how much more demanding the inversion would be with all 5 free parameters. Could this be revisited in the discussion somewhere?
L286288: As the total number of iterations is reported in the previous paragraph, could the total number of iterations be listed here as well?
L292294: It seems that the convergence behavior would be another interesting point for further exploration in the discussion.
L296301: This seems like a paragraph that belongs in the discussion section, perhaps revisiting the results of the examples that have been presented and how/why the results differ from the FEM solutions or estimates from other inversions. As the discussion section is currently fairly short, perhaps this could be moved and expanded?
L304312: This section is probably the one I would seek out as a reader and it is currently quite brief. It would be nice to expand this and add some structure (i.e., multiple paragraphs) to discuss each of the potential benefits in more detail. For instance, I would be interested to know more about the computational efficiency of this approach compared to the alternatives, and this is only briefly mentioned in the current text. My sense is that the 3D PINNs can be quite fast, so this would be important to highlight if you want to interest readers in applying this method.
L315: "CPU platform" seems a bit odd to me. Perhaps there is another term to use, such as "traditional CPUbased systems"?
Figure 2: Perhaps the units for the lower xaxes could be "Myr"?
Figure 4: My experience has been that it might be easier to see how the error varies if these plots were made as percent error, rather than temperature on the x axis. Perhaps this could be considered here.
Figure 5: Perhaps the loglog plot in panel b makes things look nicer, but it makes it a bit harder to visualize the error in predicted age. Perhaps a percent error could be considered here as well.
Figure 6: It is quite hard to see the blue curves in general, and in panel a, it seems the value of the total loss is still decreasing for the green model. Is there any reason to think the loss would not decrease more for that case?
Figure 7: The subplot labels are hard to see in some plots. Perhaps the background could be solid white?
Figure 10: The lower xaxes could again use "Myr" for their unit.
Figure 13: Could these be plots using percent difference?
Figure 14: It might be easier to compare the models if plotted as percent difference here (again). It would also require only two panels. And I would encourage using a colormap that is safe for readers with color blindness (e.g., https://www.fabiocrameri.ch/colourmaps/).
Figure 17: Again, "Myr" could be used here.
Technical comments
L139: This line should not be indented.
L212: "Figure 9" misspelled in citation.
L296: "...with Braun and Robert..."
Citation: https://doi.org/10.5194/gchron202324RC2 
AC1: 'Reply on RC2', Ruohong Jiao, 23 Jan 2024
1 We will add more text to explain the benefit of the method. Reviewer 1 also pointed this out.
2 We will add more detailed introduction of neural networks.
3 We will separate the method and result sections where possible.
4 We will add percentages in the comparisons of temperature solutions between the PINN and FE.
5 We will elaborate the text where necessary, as noted by the reviewer.

Linebyline responses
L111 Highlight of PINNs will be added. Dabie Shan has a well studied postorogenic history, simple tectonic setting, and has been used as a site for applying or testing different modelling techniques. Abstract will be modified.
L1224 Good point. This will be a focus for revising the manuscript. Reviewer 1 also raised the same point.
L3132 The statement will be rewritten to avoid confusion.
L3537 Other factors that affect the reliability of AER will be added. Thanks for the reference.
L4041 Agreed. The current paragraph has caused some confusion, and I agreed that it is worth to introduce the Tt history modelling techniques. The paragraph will be rewritten for clarification.
L52 Title will be changed to “calculating crustal temperatures”.
L6472 I will check Tc1D and add it here.
L91 Agreed. I will emphasise this in the revision.
L9397 A short introduction of neural networks will be added.
L117 Good idea. I will do that.
L131135 I will improve the writing here. L_f is calculated on all collocation points with the governing PDEs, equation 1 in this case.
L148149 The spikes indicate that the learning rate was set too high. In the revision, I could set the a stepwise learning rate to remove or reduce the occurrence of such spikes.
L156157 I will add the error in percentage forms.
L163164 I will add the error in percentage forms.
L177189 Thanks. The methods in other sections will be elaborated. I will separate the method and result sections in each model experiment.
L208 Yes. I will add that.
L209 Yes. Reviewer 1 also thinks it is confusing here. I want to say that the solutions (exhumation histories) to the synthetic data may not be unique for the slow exhumation period. I will verify this by adding result using numerical methods, and compare it to that from PINNs+ DE.
L213 I will do that, emphasising the challenges of 3D models.
L235237 Good suggestion. I will rearrange the text and use “Myr” to indicate model time.
L260 I will remove it.
L264, 270271 Percentage form of norms will be added.
L273 It will be changed to “To test the capability of PINNs in solving an inverse problem, we try to solve the posorogenic uplift history of Dabie Shan using published lowtemperature thermochronological ages.”
L276277 I will explore this in the inversion, but exploration of the 5D space could be time consuming with my current computing power.
L286288 Yes. I will add that. It’s 1,800.
L292294 I will investigate the behaviour of DE and add it in discussion.
L296301 I will investigate and discuss this misfits in more detail. One possible way is to rerun the Pecube modelling without the isostatic compensation component.
L304312 I will elaborate the advantage or potentials of PINNs in the revision. Reviewer 1 also mentioned that this could be added to the instruction. I will carefully revise and make sure the rewriting will be in detail but not repetitive between introduction and discussion.
L315 Will change to “CPUbased system”
Figure 2 Yes. It will be changed.
Figure 4 Good idea. I will add panels to show the percentage.
Figure 5 I will add that.
Figure 6 The models will be run longer to get a more stable loss curve.
Figure 7 I will change the background colour to white.
Figure 10 Yes. It will be changed.
Figure 13 It is possible. I will add plots for better comparison of the results.
Figure 14 I will add panels showing the difference and change the colormap.

Technical comments
L139, L212, L296 Typos will be corrected.
Citation: https://doi.org/10.5194/gchron202324AC1
Model code and software
PINNs for thermochronology Ruohong Jiao https://github.com/jiaor/PINNs_Chron/
Viewed
HTML  XML  Total  BibTeX  EndNote  

319  115  22  456  13  14 
 HTML: 319
 PDF: 115
 XML: 22
 Total: 456
 BibTeX: 13
 EndNote: 14
Viewed (geographical distribution)
Country  #  Views  % 

Total:  0 
HTML:  0 
PDF:  0 
XML:  0 
 1