printf("by %1$s on %2$s", 'string1', 'string2'); does not work, whereas printf('by %1$s on %2$s', 'string1', 'string2'); does.

I am really creating a Wordpress theme, and following a original twentyten theme very carefully. The strange factor is, I have used double quotes on my previous printf() claims with no problems whatsoever.

It is crucial to understand php is dealing with single cited and double cited strings in a different way.

Read more in official php docs, but allow me to provide you with a highlight:

$t = 'bla';
echo '$t';

will output $t, where

$t = 'bla';
echo "$t";

will output bla

Because the other solutions say, it goodies $s like a variable, you can always escape the $

printf("by %1\$s on %2\$s", 'string1', 'string2');

I'd however use single quotes, as php does not have to parse the string and it is therefore faster.

Like you will find the '$s' bit inside your string. When utilizing double quotes PHP translates it as being a flexible and attempts to parse it. You most likely used double quotes with no $ inside it earlier.

Because when you're using double quotes the $s is treated like a variable

As with:

$x = "World";
echo "Hello $x"; // Will print: "Hello World

while when utilizing:

$x = "World";
echo 'Hello $x'; // Will just print "Hello $x"

For any more in depth explanation you should check the manual:

Strings in General

Single quoted versus Double quoted