Solving the inverse problem for lattice 2-point and 4-point correlators