Stochastic processes such as stochastic differential equations (SDEs) and Gaussian processes are used as statistical models in many disciplines.  However, there are many situations in which a statistical design or inference problem associated with these processes is intractable, and approximations are then required.  Traditionally these approximations often come without measures of quality. We motivate using three examples:

(i) Approximating intractable likelihoods for SDEs;

(ii) Using "near-optimal design" to find spatial designs that minimize integrated mean square error;

(iii) Using well-designed data subsets to enhance stochastic gradient descent (SGD) for big data statistical learning.

We demonstrate approaches to framing such problems using a statistical perspective so that we can probabilistically quantify uncertainties when making approximations.  Depending on the problem, we achieve this using a range of modern statistical methods such as Gaussian processes, point processes, sequential design, and quantile regression. This is joint research with Ge Liu, Sophie (Huong) Nguyen, Grant Schneider, Radu Herbei, Devon (Chunfang) Lin, and Matthew Pratola.