Melt butter in a Dutch oven over medium heat.
Add onion and celery and cook until soft, about 5 minutes.
Add garlic and cook 1 minute.
Sprinkle flour on top. Stir in and cook for 1 minute.
Gradually whisk in beef broth and bring to a simmer.
Stir in half-and-half, Thousand Island, horseradish, Worcestershire sauce, caraway seeds, and cayenne pepper. Bring to a simmer for 5 minutes.
Turn heat to low and stir in sauerkraut, 1 1/2 cups swiss cheese, and corned beef. Season to taste with salt and pepper. Remove from heat and keep warm.
Cut rye bread into squares. (I used a round cookie cutter to make circles. Place on baking sheet and sprinkle with remaining swiss cheese. Place in 400 degree oven until cheese is melted.
Serve soup with croutons.