@Morten:

This would go within your theme template.php or module file and would need to be wrapped in a hook_form_comment_form_alter() function.

For example:

function THEMENAME_form_comment_form_alter(&$form, &$form_state) {
$form['comment_body'][$form['comment_body']['#language']][0]['#attributes']['placeholder'] = t('YOUR TEXT');
}