Please try adding the settings app to the Kiosk policy.
Navigate to Policies -> Kiosk Lockdown-> Android Kiosk Lockdown-> Multi-App kiosk, click on the ‘+’ icon and select the Settings app. You can search for the app by its bundle identifier using the search tab at the top right of the window.
Associate targets and save the policy.
Please use the help link to know more about the multi-app kiosk.
To find the bundle ID of an application, look for the app in Google Play Store using a web browser. At the end of the URL, after the ‘id=’, will be listed the bundle ID.
Please use the below screenshot for a reference:
Also, there are many apps in the Play Store that will help you find the package name IDs of the apps published in the play store.