The Supreme Court on Monday refused to stay the "scientific survey" of the Bhojshala complex, a medieval-era structure that both Hindus and Muslims claim as their own in Madhya Pradesh's Dhar district, but said no action should be taken without its permission on the outcome of the ASI survey.