Casualties are usually removed from the back of a unit's formation in which case the unit is left exactly as it is. Remember, a unit engaged in combat cannot move in its move phase, and so has no opportunity to change its formation.
If a unit is fighting in a single rank, forming a line of troops, then casualties are removed from either end. If this results in an enemy attacking the side of the formation becoming separated from the combat, then compensate for this by moving one or both units so that they remain in contact where possible.
If a character model is fighting as part of a unit's formation, perhaps leading the unit into battle, then his death will leave a gap in your front rank. In this case move a model forward from the rear rank to fill the gap, or, if the unit is fighting in a single rank, move a model from one edge. See the section on Heroes and Wizards for more about character models.