(?) marks in HTML. Encoding issue from content from the database?

Question

Any idea why this is happening?

It looks to be happening mainly with apostrophes and hyphens. Any ideas if I can fix this? I pull the data from my database and print it to the page like:

<div class="block">
              <?=$details['agenda'] ?>
</div>

Answer 1

As other commenters may have mentioned, this is a character encoding problem. If you're lucky, you can force your HTML page to render in UTF-8 and that will resolve it.

Unfortunately, if you're not lucky, you'll discover that the characters are stored in the database in the wrong encoding. Or maybe the database converts them. Or maybe the character encoding data has been destroyed along the path! There's no way of knowing in advance where those characters have been damaged.

The best way I know to fix problems like this is to force every step along your path to follow UTF-8 content encoding. For example, you probably go through steps like this:

1. Content author writes a document in Microsoft Word containing "SmartQuotes"
2. Content author copies-and-pastes into the edit box of a content management system.
3. Content management system saves to the database.
4. Database may or may not store data in Unicode internally - make sure you use nvarchar (or whatever unicode type your database supports).
5. Reading from the database may need to scan for characters.

However, it's very tricky to fix this! A long time ago, I used to have a habit of writing "detect-and-fix" routines like this:

$smartquotes = array("”", "“");
str_replace($smartquotes, '"', $mytext);

Of course you know what the problem is - I'd keep discovering new characters I had to fix. Microsoft Word likes to do tons of unusual characters - copyright, registration marks, apostrophes, hyphens, and so on. I'd keep adding to this function, over and over, until I went crazy. So nowadays I just go through my entire content delivery path and force everything to obey UTF-8 rules; that seems to resolve it in most cases.

Good luck!