This talk focuses on large-scale bilevel optimization problems that exhibit a natural block-separable structure, a setting that frequently appears in modern machine learning pipelines, large data-cleaning tasks, and parametric model selection. Such problems pose substantial computational challenges due to the interaction between high-dimensional upper-level variables and the need to repeatedly solve lower-level optimization problems. To address these difficulties, we propose a randomized block-proximal framework that updates only a small subset of hyperparameters at each iteration, thereby reducing both memory usage and computational cost. The method relies on implicit differentiation to compute hypergradients, but does so in a scalable, matrix-free manner that avoids forming or storing large Hessian matrices. This combination yields an approach that is well-suited for problems with large number of parameters and offers a practical alternative to traditional full-batch bilevel solvers.