Mormon.org has the following to teach us about seeking repentance so that we can relieve those feelings of guilt:
"Having faith in Jesus Christ makes us want to live good lives. When we sin and seek to repent, we recognize and feel deep and sincere sorrow for what we’ve done wrong. God understood when He created the earth that we wouldn’t be perfect, so He provided a way for us to overcome our sins. The ability to repent is truly one of our greatest blessings.
"To repent we need to recognize and feel sorry for what we’ve done wrong, do whatever is possible to repair the damage it may have caused, and to leave our sinful behavior behind. Repentance can be difficult and requires a lot of honesty, but the joy and freedom we feel when we turn away from our sins are well worth the effort. Because Christ suffered for our sins, we can be forgiven when we repent. That’s why the Atonement is so important to all of us.
"We believe Christ's Atonement gives us the ability to repent and become clean from sin. To say we have to repent for our sins may sound like a punishment, but the real punishment is the guilt, sorrow, and disappointment we feel when we sin. Repentance is the opposite of punishment, then, because it allows us to become clean in God’s eyes and remove the feelings of guilt that follow our bad choices."
In Alma 34, it says:
14 And behold, this is the whole meaning of the law, every whit pointing to that great and last sacrifice; and that great and last sacrifice will be the Son of God, yea, infinite and eternal.
15 And thus he shall bring salvation to all those who shall believe on his name; this being the intent of this last sacrifice, to bring about the bowels of mercy, which overpowereth justice, and bringeth about means unto men that they may have faith unto repentance.
16 And thus mercy can satisfy the demands of justice, and encircles them in the arms of safety, while he that exercises no faith unto repentance is exposed to the whole law of the demands of justice; therefore only unto him that has faith unto repentance is brought about the great and eternal plan of redemption.
17 Therefore may God grant unto you, my brethren, that ye may begin to exercise your faith unto repentance, that ye begin to call upon his holy name, that he would have mercy upon you;
It is my testimony that our faith in our Savior leads us to want to repent. When we fully have faith in our savior, we recognize that it is only through him that we will be able to return to our Heavenly Father. This then leads us to realize that we must be willing to repent so that we may feel clean from sin. This is an amazing gift Jesus Christ has given us. I know that if we humbly repent, our Heavenly Father forgives us. I am so grateful to have felt of the cleansing power of the atonement! It is real, and you can know for yourself!