Current processing techniques of polarization-sensitive optical coherence tomography (PS-OCT) can recover the tissue’s local, i.e. depth-resolved scalar retardance and optic axis orientation. However, system-induced polarization mode dispersion (PMD) and the presence of speckle in the measured tomograms complicate reconstruction and result in a detrimental trade-off with spatial resolution. We speculate that a machine learning approach should work well for generating an improved reconstruction. By training the model on simulated tomograms that encode the forward model and include system PMD and noise, and by testing the algorithm on experimentally acquired PS-OCT data, we aim to demonstrate a generalized PS-OCT reconstruction tool.