In this paper, we propose a framework for automatically segmenting the lumen boundary in IV-OCT with minimum time‐complexity. The framework uses OCT imaging physics based graph representation of signals and random walks image segmentation approaches. First, each OCT frame is modelled as a 4-connected graph and edge weights are assigned incorporating OCT signal attenuation physics models. Second, optical backscattering maxima is tracked along each A-scan of OCT and is subsequently refined using global gray‐level statistics and used for initializing seeds. This automates the seed selection process thus avoiding manual interaction. Finally, lumen boundary is segmented using the random walks image segmentation using the initialized seeds.