FEAT: Scenario DatasetConfiguration

[rlundeen2]

This PR introduces DatasetConfiguration class that is passed to scenarios in initialize_async to address several pain points.

Big available default datasets

It was tough to change the default datasets - some were too big and some too small. As an example, Foundry used harm_bench, which is too big for the default. So it would randomly select 4 by default. But how could users run against all 100? There was no way to configure this. On the flip side garak.encoding_scenario had a large dataset that took a very long time to run, but there was no way to make it small by default and still have the entire dataset available.

This change allows scenarios to be configured with a default (e.g. name=harm_bench, max=4). But users can easily configure both the dataset name or max differently.

It also helps with our end to end tests, which were taking a million years because encoding scenario had so many datasets (this pr reduces to only 3 by default)

Allows dataset params from the front end

This also allows users to specify the dataset names they want to use from pyrit_scan and pyrit_shell. Previously, users had to use the dataset defaults.

--dataset-names DATASET_NAMES [DATASET_NAMES ...]
Dataset names to load for the scenario (overrides scenario defaults)
--max-dataset-size MAX_DATASET_SIZE
Maximum number of seed groups to use (randomly samples if dataset is larger)

Deprecates Incompatible Parameters

objective, seed_prompts are now deprecated as initialization parameters for scenarios

Tests

• Unit tests added!
• End to end tests passed 2c281be

[romanlutz]

Suggested change

	# Check for unexpected response type (e.g., API returned a string during content filter)
	if not hasattr(response, "choices"):
	raise PyritException(
	message=f"Unexpected response type from API: {type(response).__name__}. "
	f"Expected ChatCompletion object with 'choices' attribute. Response: {str(response)[:200]}"
	)
	# Check for missing choices
	if not response.choices:
	raise PyritException(message="No choices returned in the completion response.")
	# Check for missing choices
	if not hasattr(response, "choices") or not response.choices:
	raise PyritException(message="No choices returned in the completion response.")

[romanlutz]

Maybe I've taken one too many combinatorics classes but we should probably specify whether this will be sampling with/without replacement.