I have just used my application to heroku and pointed my custom domain towards the heroku servers. How do i look into the records during my heroku database?

You should use heroku console and check out your records with Model.all or other method.

If you wish to backup the database take a look at heroku PG backup copies, after this you can import your database in your local machine and check out it there. Based on your db adapter you could utilize sqlite browser for sqlite3 or phpmyadmin for MySQL.

heroku db:pull to drag your production DB in your area to consider a look inside it.