Skip to content

Cholesky decomposition fails for ATLAS_2JET_7TEV_R06 in a closure test

Created by: andreab1997

When running a level-2 closure test which contains the dataset ATLAS_2JET_7TEV_R06, it fails (during n3fit execution) with the error numpy.linalg.LinAlgError: 65-th leading minor of the array is not positive definite

Reproduce

A minimal runcard to reproduce this bug is the following

#
# Configuration file for NNPDF++
#

############################################################
description: NNPDF4.0 NNLO Global, closure test

############################################################
# frac: training fraction
# ewk: apply ewk k-factors
# sys: systematics treatment (see systypes)
dataset_inputs:
- {dataset: ATLAS_2JET_7TEV_R06, frac: 0.75, cfac: []} 

############################################################
datacuts:  
  t0pdfset: 231223-ab-baseline-nnlo-global-NNLOcuts_iterated     # PDF set to generate t0 covmat
  q2min: 3.49                       # Q2 minimum
  w2min: 12.5
  use_cuts: internal
############################################################
theory:
  theoryid: 708         # database id

############################################################
trvlseed: 1727532335
nnseed: 1737785873
mcseed: 2123509817
genrep: true          # true = generate MC replicas, false = use real data

parameters: # This defines the parameter dictionary that is passed to the Model Trainer
  nodes_per_layer: [25, 20, 8]
  activation_per_layer: [tanh, tanh, linear]
  initializer: glorot_normal
  optimizer:
    clipnorm: 6.073e-6
    learning_rate: 2.621e-3
    optimizer_name: Nadam
  epochs: 17000
  positivity:
    initial: 184.8
    multiplier:
  integrability:
    initial: 10
    multiplier:
  stopping_patience: 0.1
  layer_type: dense
  dropout: 0.0
  threshold_chi2: 3.5

fitting:
  fitbasis: EVOL  # EVOL (7), EVOLQED (8), etc.
  basis:
  - {fl: sng, trainable: false, smallx: [1.091, 1.119], largex: [1.471, 3.021]}
  - {fl: g, trainable: false, smallx: [0.7795, 1.095], largex: [2.742, 5.547]}
  - {fl: v, trainable: false, smallx: [0.472, 0.7576], largex: [1.571, 3.559]}
  - {fl: v3, trainable: false, smallx: [0.07483, 0.4501], largex: [1.714, 3.467]}
  - {fl: v8, trainable: false, smallx: [0.5731, 0.779], largex: [1.555, 3.465]}
  - {fl: t3, trainable: false, smallx: [-0.5498, 1.0], largex: [1.778, 3.5]}
  - {fl: t8, trainable: false, smallx: [0.5469, 0.857], largex: [1.555, 3.391]}
  - {fl: t15, trainable: false, smallx: [1.081, 1.142], largex: [1.491, 3.092]}

############################################################
positivity:
  posdatasets:
  - {dataset: POSF2U, maxlambda: 1e6}        # Positivity Lagrange Multiplier
  - {dataset: POSF2DW, maxlambda: 1e6}
  - {dataset: POSF2S, maxlambda: 1e6}
  - {dataset: POSFLL_19PTS, maxlambda: 1e6}
  - {dataset: POSDYU, maxlambda: 1e10}
  - {dataset: POSDYD, maxlambda: 1e10}
  - {dataset: POSDYS, maxlambda: 1e10}
  - {dataset: POSF2C_17PTS, maxlambda: 1e6}
  - {dataset: POSXUQ, maxlambda: 1e6}        # Positivity of MSbar PDFs
  - {dataset: POSXUB, maxlambda: 1e6}
  - {dataset: POSXDQ, maxlambda: 1e6}
  - {dataset: POSXDB, maxlambda: 1e6}
  - {dataset: POSXSQ, maxlambda: 1e6}
  - {dataset: POSXSB, maxlambda: 1e6}
  - {dataset: POSXGL, maxlambda: 1e6}

############################################################
integrability:
  integdatasets:
  - {dataset: INTEGXT8, maxlambda: 1e2}
  - {dataset: INTEGXT3, maxlambda: 1e2}

############################################################

closuretest:
  filterseed: 3345348918 # Random seed to be used in filtering data partitions
  fakedata: true     # true = to use FAKEPDF to generate pseudo-data
  fakepdf: 231223-ab-baseline-nnlo-global-NNLOcuts_iterated      # Theory input for pseudo-data
  errorsize: 1.0    # uncertainties rescaling
  fakenoise: true    # true = to add random fluctuations to pseudo-data
  rancutprob: 1.0   # Fraction of data to be included in the fit
  rancutmethod: 0   # Method to select rancutprob data fraction
  rancuttrnval: false # 0(1) to output training(valiation) chi2 in report
  printpdf4gen: false # To print info on PDFs during minimization

############################################################
debug: false
maxcores: 8

Things to note

  • The experimental covariance matrix generated during vp-setupfit to generate the level-1 data seems to be ok. In fact, vp-setupfit does not crash.
  • In order to reproduce this bug in master, another bug needs to be solved (which is solved already in #2007, look at coredata.py).

I tried to debug this myself but for the moment I was not able to figure out the problem.

CC: @scarlehoff @RoyStegeman @comane