RATIONALE: Drugs used in chemotherapy, such as irinotecan, leucovorin, fluorouracil, and oxaliplatin, use different ways to stop tumor cells from dividing so they stop growing or die. Radiation therapy uses high-energy x-rays to damage tumor cells. Combining radiation therapy with chemotherapy may kill more tumor cells.

PURPOSE: This phase II trial is studying how well different regimens of induction chemotherapy followed by chemoradiotherapy work in treating patients with locally advanced adenocarcinoma of the rectum.

OBJECTIVES:

• Determine the feasibility of obtaining a pre-treatment determination of intratumoral molecular markers (TS, DPD, and ERCC-1) for use in selection of the appropriate regimen for induction cytotoxic combination chemotherapy in patients with cT3-4 rectal adenocarcinoma.
• Determine the response probability (unconfirmed, complete and partial) in patients treated with targeted induction cytotoxic chemotherapy.
• Determine the toxicity of targeted induction cytotoxic chemotherapy and chemoradiotherapy in these patients.
• Determine the response probability in these patients treated with chemoradiotherapy.

OUTLINE: This is a multicenter study.

• Induction chemotherapy: Patients are assigned to 1 of 3 treatment groups based on molecular analysis of the pretreatment tumor specimen.

  • Group I (lower likelihood of resistance to a fluorouracil-based regimen): Patients receive irinotecan IV over 90 minutes and leucovorin calcium IV over 2 hours on day 1 and fluorouracil IV over 46 hours beginning on day 1.
  • Group II (higher likelihood of resistance to a fluorouracil-based regimen): Patients receive oxaliplatin IV over 2 hours and irinotecan IV over 90 minutes on day 1.
  • Group III (high likelihood of sensitivity to oxaliplatin and fluorouracil therapy): Patients receive oxaliplatin IV over 90 minutes and leucovorin calcium IV over 2 hours on day 1 and fluorouracil IV over 46 hours beginning on day 1.

Treatment in all groups repeats every 2 weeks for 4 courses in the absence of disease progression or unacceptable toxicity.

Patients are evaluated for response approximately 2 weeks after the completion of induction chemotherapy. Patients with stable disease or better receive chemoradiotherapy.

• Chemoradiotherapy: Beginning approximately 3 weeks after the completion of induction chemotherapy, patients receive oral capecitabine twice daily continuously for 5 weeks and concurrent radiotherapy once daily 5 days a week for 5 weeks.

After chemoradiotherapy, patients may undergo attempted surgical resection at the discretion of the treating physician.

Patients are followed every 3 months for 1 year and then every 6 months for 2 years.

PROJECTED ACCRUAL: A total of 10-65 patients will be accrued for this study.

• Drug: FOLFIRI regimen
• Drug: FOLFOX regimen
• Drug: capecitabine
• Drug: fluorouracil
• Drug: irinotecan hydrochloride
• Drug: leucovorin calcium
• Drug: oxaliplatin
• Radiation: radiation therapy

DISEASE CHARACTERISTICS:

• Histologically confirmed primary adenocarcinoma of the rectum

• Locally advanced disease (clinical T3-4, N0-2, M0) based on at least 1 of the following criteria:

  • Clinically fixed tumors on rectal examination with tumor adherent to the pelvic sidewall and/or sacrum
  • Sciatica attributed to sacral root invasion with CT scan/MRI evidence of the lack of clear tissue plane is considered evidence of fixation
  • Hydronephrosis on CT scan or intravenous pyelogram of ureteric or bladder invasion by cystoscopy and cytology or biopsy
  • Invasion into the prostate, vagina, or uterus
• Transmural penetration of tumor through the muscularis propria as evidenced by CT scan or MRI and endorectal ultrasound
• Distal border of the tumor must be at or below the peritoneal reflection (within 12 cm of the anal verge) by proctoscopic examination
• Measurable disease by x-ray, scans, or physical examination
• Available tumor tissue to determine molecular profile of the tumor before study treatment
• No clinical evidence of high-grade (lumen diameter < 1 cm) large bowel obstruction unless a diverting colostomy has been performed

PATIENT CHARACTERISTICS:

Age

• 18 and over

Performance status

• Zubrod 0-2

Life expectancy

• Not specified

Hematopoietic

• WBC ≥ 3,500/mm^3
• Absolute neutrophil count ≥ 1,500/mm^3
• Platelet count ≥100,000/mm^3

Hepatic

• Bilirubin ≤ 1.5 times upper limit of normal (ULN)
• SGOT or SGPT ≤ 2.5 times ULN
• Alkaline phosphatase ≤ 2.5 times ULN

Renal

• See Disease Characteristics
• Creatinine ≤ 1.5 times ULN OR
• Estimated creatinine clearance > 50 mL/min

Cardiovascular

• No significant cardiac disease
• No recent myocardial infarction

Gastrointestinal

• See Disease Characteristics
• Able to swallow oral medication
• No active inflammatory bowel disease

Other

• Not pregnant or nursing
• Fertile patients must use effective contraception
• No other malignancy within the past 5 years except adequately treated basal cell or squamous cell skin cancer or carcinoma in situ of the cervix
• No prior unanticipated severe reaction to study drugs
• No known dihydropyrimidine dehydrogenase deficiency
• No serious uncontrolled infection
• No other serious medical illness that would preclude study treatment

PRIOR CONCURRENT THERAPY:

Biologic therapy

• Not specified

Chemotherapy

• No prior chemotherapy for colon or rectal cancer

Endocrine therapy

• Not specified

Radiotherapy

• No prior pelvic radiotherapy
• No prior intra-operative radiotherapy or brachytherapy
• No concurrent intra-operative radiotherapy or brachytherapy
• No concurrent intensity-modulated radiotherapy

Surgery

• See Disease Characteristics
• See Radiotherapy