Pipeline spills and pollution of the environment by crude oil pose a threat to natural resources, especially soil and water. One such incident occurred on 25 September 2018 in the area of Budrovac (Croatia; 46°00′14.6″ N 17°04′16.8″ E) on agricultural land as a pipeline spill. Bioremediation of the contaminated soil was carried out with organic pollutants using an environmentally safe absorbent Spill-Sorb (Canadian Sphagnum Peat Moss) and a mineral fertilizer—nitrogen. The experiment was conducted in the greenhouse of the Faculty of Agriculture, Croatia, during a six-month (October 2018–April 2019) study. Samples of agricultural soils contaminated with total petroleum hydrocarbons (TPHs) and polycyclic aromatic hydrocarbons (PAHs) were taken after the rupture of the local gas condensate pipeline. The experiment was conducted in five treatments in triplicate: I-control (clean soil); II-100% contaminated soil + organic absorbent + nitrogen; III-100% contaminated soil + organic absorbent; IV-50% clean soil + 50% contaminated soil + organic absorbent + nitrogen; and V-50% clean soil + 50% contaminated soil + organic absorbent. The soil properties studied were pH, organic matter content, carbon and nitrogen content and ratio, and changes in the concentration of potential organic contaminants—TPHs and individual PAHs. The results demonstrated that the mixture of organic absorbent and nitrogen efficiently removed organic pollutants from the contaminated soil within six months. However, the application of Spill-Sorb alone was more effective for the degradation of hydrocarbons. The effectiveness of the absorbent studied was dependent on the concentration of organic pollutants and nitrogen application.