Evaluation of Newly Synthesized Schiff Base Pd(II) Complexes for Prostate Cancer Treatment Through In Vitro Cytotoxicity and Molecular Mechanistic Studies

1. Introduction

The treatment of cancer is the most challenging issue of modern science, particularly due to the resistance of tumor cells to many utilized protocols and treatments, as well as due to serious side effects. Metal complexes have arouses as a suitable alternative for the treatment of different ailments due to their unique properties. Among them, platinum-based drugs still represent the most explored options for cancer treatment, but the novel studies concentrate on the other metal drugs with better efficacy [1,2,3,4,5]. The idea behind the design of novel metalotherapeutics with increased activity and lowered toxicity is based mainly on the elucidation of antitumor molecular mechanisms different from those expressed by platinum compounds [5]. The last two decades have brought increased interest in the design and synthesis of palladium complexes that could serve as alternative anticancer drugs [6,7,8] inspired by their coordination mode similar to that of platinum, but, according to some studies performed on rats, significantly lower toxicity [9]. As it undergoes redox reaction, the palladium is also capable of the formation of reactive oxygen species (ROS). Despite the obvious similarity of these two metalotherapeutics groups, the palladium complexes are much more reactive, therefore, imposing the need for their stabilization through the utilization of specific chelating ligands. The choice of the suitable ligand is a crucial step in the development of the promising antitumor candidate, knowing that finely tuned physicochemical properties of used ligand, its biological potential, and the strength of the ligand-metal bond directly influence the selectivity and mechanism of the antitumor activity at a molecular level.

The Schiff bases represent molecules widely utilized in many biological studies as pharmacophores capable of forming stable complexes with a variety of metals. Besides the nitrogen atom from imino groups, the molecular scaffold of Schiff bases can be easily manipulated to contain other donor atoms, such as oxygen and Sulphur and, in that way provide the desired physicochemical or pharmaceutical properties necessary for the establishment of targeted biological activity. There are numerous reports evidencing the pharmacological potential of these compounds reflected in pronounced antitubercular, anti-inflammatory, antiviral, anticholinesterase, antibacterial, anticancer, and antioxidant activities [10,11,12,13,14,15,16,17]. When it comes to the tumor cells, Schiff bases can be hydrolyzed by tumor cells in a selective manner or provide release of active amines as antimetabolites [18]. Other antitumor effects of Schiff bases include the cycle cell arrest by the production of free radicals and damage of proteins and nucleic acids [19,20,21]. As a consequence of above mentioned, the Schiff bases are also utilized as a ligand for the design of palladium complexes as promising candidates for photodynamic therapy [22], as well as effective anticancer agents against a variety of cancer cell lines through the different molecular pathways, such as inhibition of proteasomes, enzymes, and cancer protein markers [23].

In the light of the foregoing, herein we report the synthesis, characterization, and results of in vitro cytotoxic assay of three novel palladium (II) complexes bearing Schiff base ligands combining salicylic aldehyde and tryptamine/tyramine and p-OH benzylamine structural motifs. While tryptamine and tyramine represent examples of biogenic amines, p-OH benzylamine was used due to the structural resemblance to the tyramine scaffold. The cytotoxic activity was assessed on two human prostate cancer cell lines PC-3 and DU-145 and the results were compared with those obtained with healthy fibroblast cell line MRC-5. It was of interest to compare the influence of the palladium metal ion coordination to the Schiff base ligand on their antitumor potential. To gain further insight into their possible mode of action, we focused on one of the key molecular targets in prostate cancer – the androgen receptor (AR). This receptor plays a central role in disease progression by regulating the transcription of genes involved in cell growth, survival, and differentiation upon binding with androgens such as testosterone and dihydrotestosterone [24]. Notably, AR signaling remains active even under androgen-deprived conditions in tumor cells, facilitating continued tumor proliferation and the emergence of therapy resistance [25]. Given its critical role, a molecular docking study was undertaken to evaluate the interaction of the synthesized palladium complexes with AR, aiming to elucidate their potential as modulators of androgen receptor activity and their contribution to the observed cytotoxic effects.

2. Results and Discussion

2.1. Synthesis of Pd Complexes 2a-c

The targeted Pd(II) complexes 2a-c have been prepared in one pot procedure comprising the reaction between corresponding imine 1a-c as ligands and K₂PdCl₄ as a metal source in acetonitrile, at room temperature in a 1:1 ratio (ligand vs. metal salt), as indicated in Scheme 1. Besides the salicylic part, the ligands were composed of imine patterns – tryptamine, tyramine, and p-OH benzyl moiety structurally related to the tyramine molecular scaffold. All complexes are new compounds obtained in the form of powders and fully characterized by the IR, ¹H NMR, ¹³C NMR, mass analysis, and elemental analysis (Experimental part, Supplementary material).

2.2. NMR Studies

A detailed overview of the NMR characterization data for all synthesized compounds is provided in Supplementary Information (SI), while representative examples are discussed herein to highlight key structural changes upon complexation. The representative ¹H NMR spectra of palladium complex 2b were analyzed by comparison with the spectra of ligand compound 1b, as depicted in Figure 1, a). The azomethine proton –CH=N from the Schiff base located at 8.45 ppm is, in the case of the palladium complex, shifted to the 7.88 ppm, confirming the involvement of the imino group in the coordination with the palladium metal ion. The –OH proton peaks from salicylic moiety appearing at 13.54 ppm in the ligand spectra, completely disappear in the case of palladium complex spectra, undoubtedly confirming the coordination of palladium ion with phenolate anion. Additionally, subtle changes in the aromatic region (6.5–8.0 ppm) further support coordination-induced electronic redistribution. The multiplet pattern of aromatic protons becomes slightly broadened and shifts marginally, which can be attributed to changes in magnetic environment and restricted rotation upon complexation. Taken together, these spectroscopic features strongly support the formation of a stable square planar Pd(II) complex, in which the ligand coordinates through both imine nitrogen and deprotonated phenolic oxygen, in agreement with the proposed structure shown in the figure.

In the ¹³C NMR spectra, clear changes in the chemical shifts of characteristic carbon atoms were observed upon complexation with Pd(II), supporting the proposed coordination mode. In the free ligand 1b, the signal for the azomethine carbon (–CH=N) appears at 165.82 ppm, while the phenolic carbon bearing the –OH group (–C–OH) resonates at 155.68 ppm (Figure 2). Upon complex formation, the azomethine carbon signal shifts slightly upfield to 163.54 ppm, indicating a decrease in electron density due to the coordination of the imine nitrogen to the palladium center, which is consistent with the formation of a Pd–N bond. The phenolic carbon signal shows a negligible shift (from 155.68 ppm to 155.72 ppm), which suggests that while deprotonation of the hydroxyl group occurs, the local electronic environment of the carbon remains largely stabilized, possibly due to resonance delocalization within the chelate ring and Pd–O bond formation. Additionally, several aromatic carbon signals appear in the 120–140 ppm region, with slight shifts compared to the free ligand, reflecting electronic redistribution upon coordination. These observations provide further evidence of ligand binding and the resulting structural reorganization. Overall, the ¹³C NMR data, in conjunction with the ¹H NMR results, strongly support the successful coordination of ligand 1b to Pd(II) through the imine nitrogen and phenolate oxygen atoms in complex 2b.

The corresponding ¹H NMR spectral data for the CH=N and –C-OH protons, as most relevant shifts in complex structure determination for all complexes 2a-c are given in Table 1.

2.3. IR Analysis

The experimental IR spectra of all investigated ligands and their corresponding Pd(II) complexes are provided in the Supplementary Information (SI). IR spectral analysis offers valuable insights into the coordination behavior of ligands during complex formation with Pd(II) ions. One of the most significant vibrational bands in the FTIR spectra of Schiff base ligands is the imine stretching band (νC=N), typically observed in the range of 1610–1650 cm⁻¹ . This functional group plays a key role in metal coordination, and any shift in its position upon complexation serves as strong evidence of such interaction.

In the IR spectra of the free ligands 1a–1c, the νC=N stretching bands were observed at 1641, 1631, and 1630 cm⁻¹, respectively. After coordination with Pd(II), these signals were shifted to lower frequencies – 1638 cm⁻¹ for complex 2a, 1611 cm⁻¹ for 2b, and 1618 cm⁻¹ for 2c. This redshift, particularly pronounced in the case of complex 2b (Δν = 20 cm⁻¹), indicates a partial loss of double bond character in the C=N bond as a result of electron donation from the imine nitrogen to the palladium ion.

The extent of the νC=N shift correlates with the strength of azomethine nitrogen coordination to the Pd(II) center, further supporting its direct involvement in complex formation. The uniform trend observed across all examined systems confirms that the ligands coordinate with the palladium ion via the imine nitrogen.

These IR findings are fully consistent with the ¹H and ¹³C NMR data previously discussed, providing complementary experimental evidence for the formation of bidentate N, O-chelated palladium(II) complexes.

2.3. Mass Analysis

The electrospray ionization mass spectrometry (ESI-MS) analysis was performed to confirm the molecular composition and stoichiometry of the synthesized palladium(II) complexes. In the mass spectrum of complex 2a, a dominant signal was observed at m/z = 559.0857, which corresponds to the protonated molecular ion [M + H] ⁺, confirming the formation of the proposed bis-Schiff base Pd(II) complex (ESI, Figure SI2). Similarly, for complex 2c, a strong peak at m/z = 633.1490 was detected, also corresponding to the [M + H] ⁺ species (ESI, Figure SI5). The presence of these molecular ion peaks, which match the calculated molecular masses of the respective complexes, confirms the proposed coordination of two Schiff base ligands to the Pd(II) ion in a 2:1 ligand-to-metal ratio. These results provide direct evidence for the successful formation of the target Pd(II) complexes and are in full agreement with the spectroscopic data (NMR and IR), thereby supporting the proposed coordination environment illustrated in the structural models.

2.4. In Vitro Cytotoxic Study

The cytotoxicity of the three synthesized Pd(II) complexes (2a–c) was assessed in vitro using human prostate carcinoma cells lacking prostate-specific antigen (PSA) and androgen receptor independence (DU-145), human prostate carcinoma cells (PC-3), and non-cancerous human fibroblasts (MRC-5). The evaluation was conducted via the MTT assay following 24, 48, and 72 hours of treatment. For comparative purposes, the cytotoxic profile of cisplatin was also examined. The findings unequivocally demonstrate that all Pd(II) complexes, as well as cisplatin, exhibit dose-dependent cytotoxic effects against DU-145 and PC-3 cell lines. The IC₅₀ values for these Pd(II) complexes and cisplatin are summarized in Table 2, reinforcing their potential as anticancer agents (Figure 3).

All tested Pd(II) complexes, as well as cisplatin, significantly reduced the viability of DU-145 and PC-3 prostate carcinoma cell lines following 24, 48, and 72 hours of treatment. Among the synthesized complexes, compound 2c exhibited the most pronounced cytotoxic activity across all exposure durations and against both cell lines. Notably, the cytotoxic effect of complex 2c surpassed that of cisplatin after 48 and 72 hours of treatment under identical experimental conditions (Figure 3 and Table 3). Furthermore, while all Pd(II) complexes demonstrated a relatively lower cytotoxic impact on non-cancerous human fibroblasts (MRC-5) compared to cisplatin, exceptions were observed for complexes 2b and 2c at the 24-hour mark. These findings underscore the selective anticancer potential of the newly synthesized Pd(II) complexes (2a–c), particularly their preferential activity against human prostate cancer cell lines.

The subsequent phase of our investigation focused on determining the mode of cell death induced by the Pd(II) complexes in DU-145 and PC-3 prostate carcinoma cells. To achieve this, an annexin V/PI staining assay was employed. The results revealed that all tested Pd(II) complexes effectively triggered apoptosis (Figure 4). Notably, in DU-145 cells, treatment with an IC₅₀ dose of complex 2c resulted in the highest proportion of necrotic cells. Further analyses were conducted to investigate whether these Pd(II) complexes affected the intracellular levels of the anti-apoptotic protein Bcl-2 and the pro-apoptotic protein Bax. Additionally, caspase-3 activation was assessed in DU-145 cells exposed to the Pd(II) complexes. The findings demonstrated a significant reduction in Bcl-2 levels and a concomitant increase in BAX concentrations in cells treated with IC50 doses of the Pd(II) complexes compared to untreated controls (Figure 4). Moreover, a marked elevation in the percentage of cells exhibiting active caspase-3 was observed following treatment with the Pd(II) complexes (Figure 5). Collectively, these results indicate that the Pd(II) complexes (2a–c) decreased the Bcl-2/BAX ratio, thereby promoting caspase-3 activation and inducing apoptosis via intrinsic apoptotic pathways.

The induction of apoptosis and/or cell cycle arrest is a well-established mechanism for reducing cancer cell viability[26,27]. Notably, cancer cells exhibit a heightened reliance on DNA damage repair mechanisms and the abrogation of the G2 checkpoint. Arrest in the G0/G1 phase typically halts cellular proliferation while allowing for the repair of damage caused by anticancer agents. In contrast, G2/M phase arrest is often associated with the initiation of apoptosis. In our study, we investigated the effects of IC₅₀ concentrations of Pd(II) complexes 2a–c on the cell cycle of DU-145 prostate cancer cells 24 hours post-treatment. Flow cytometric analysis was performed on propidium iodide (PI)-stained cells to evaluate cell cycle distribution (Figure 6). The results revealed that all Pd(II) complexes induced G0/G1 phase arrest in DU-145 cells, highlighting their ability to interfere with cell cycle progression. Furthermore, variations in the mechanisms of action among the Pd(II) complexes (2a–c) were observed, underscoring their distinct biological activities.

The long-term antiproliferative effect of Pd(II) complexes, was tested by using clonogenic assay. All tested comounds showed significant antiproliferative effect which was dose-dependend, and the most pronounced effect was detected after treatment with complex 2c (Figure 7).

The query outlines the synthesis and evaluation of three novel Pd(II) complexes (2a–2c) for their cytotoxicity and mechanisms of action against human prostate carcinoma cells (DU-145 and PC-3) and non-cancerous fibroblasts (MRC-5). The findings demonstrate that all three complexes exhibit potent and selective cytotoxic effects on prostate carcinoma cells, with complex 2c showing superior efficacy against DU-145 cells after 72 hours compared to cisplatin. The cytotoxic potency follows the order 2a < 2b < 2c, potentially due to the presence of an indole ring in the ligand of complex 2c [28,29]. Conversely, the selectivity increases in the reverse order (2c < 2b < 2a), with all Pd(II) complexes displaying greater selectivity than cisplatin.

The search results corroborate the potential of Pd(II) complexes as anticancer agents. For instance, studies have shown that Pd(II) complexes induce apoptosis through mechanisms such as mitochondrial membrane depolarization, Bax upregulation, and Bcl-2 downregulation in prostate cancer cells like DU-145 and PC-3 [30,31]. Additionally, some Pd(II) complexes demonstrate selective cytotoxicity against cancer cells while sparing non-cancerous cells. The observed differences in efficacy and selectivity among Pd(II) complexes may be attributed to structural variations in their ligands, as highlighted in other research on similar compounds. These findings underscore the therapeutic potential of Pd(II) complexes for prostate cancer treatment.

Pd(II) complexes exhibit substantial anticancer potential against DU-145 human prostate cancer cells, primarily through the induction of apoptosis, particularly early apoptosis [32]. This mechanism was confirmed in our study, where treatment with Pd(II) complexes resulted in a significant transition of cells from viability to the early apoptotic stage. These complexes demonstrate both cytotoxic and antiproliferative properties by disrupting cellular metabolism and inducing cell death [30]. Furthermore, Pd(II) complexes exert both short-term and long-term antiproliferative effects, significantly impairing cell growth and abolishing long-term proliferation. Their action also includes inhibition of cell migration and adhesion, induction of morphological changes such as cellular shrinkage, and interaction with DNA, leading to double-stranded DNA cleavage – a key contributor to their cytotoxic effects [33].

Studies corroborate these findings, highlighting that Pd(II) complexes induce apoptosis via mitochondrial membrane depolarization, Bax upregulation, and Bcl-2 downregulation [8]. Additionally, their ability to disrupt DNA integrity without activating DNA repair enzymes further emphasizes their therapeutic potential [34]. These results collectively position Pd(II) complexes as promising candidates for prostate cancer treatment due to their multifaceted anticancer mechanisms.

Research has demonstrated that Pd(II) complexes can induce apoptosis through multiple pathways, including the activation of cell death receptors and the generation of reactive oxygen species (ROS). These complexes have shown significant efficacy against DU-145 prostate cancer cells, highlighting their potential as anticancer agents. The metabolic effects triggered by Pd(II) complexes can be more rapid compared to those of cisplatin, which may result in faster patient recovery. This rapid metabolic response is advantageous as it could potentially reduce the duration and severity of side effects associated with chemotherapy.

Studies on various Pd(II) complexes have revealed diverse mechanisms of action, including DNA interaction, ROS-mediated mitochondrial dysfunction, and the activation of intrinsic and extrinsic apoptotic pathways [8,35]. The ability of Pd(II) complexes to induce apoptosis without relying on DNA damage, as seen with cisplatin, suggests they may offer a distinct therapeutic approach with potentially fewer side effects [35]. Additionally, the activation of cell death receptors and the induction of apoptosis through ROS production underscore the versatility of Pd(II) complexes in targeting cancer cells [8,36].

Palladium(II) complexes have demonstrated significant cytotoxic and growth-inhibitory effects on the DU-145 human prostate adenocarcinoma cell line, primarily through mechanisms such as DNA interaction and apoptosis induction. These complexes induce cellular changes, including reduced cell numbers and morphological alterations, which are indicative of their cytotoxic effects. The formation of DNA adducts following the interaction of Pd(II) complexes with DNA is a key contributor to their anticancer activity. Studies highlight that complexation with thiosemicarbazone ligands enhances the antitumor potential of Pd(II) complexes, making this a promising strategy for developing novel anticancer agents. For instance, Pd(II)-thiosemicarbazone complexes have shown potent cytotoxicity against DU-145 cells, with IC₅₀ values significantly lower than those of cisplatin [37]. Furthermore, Pd(II) complexes have been shown to modulate protein expression in DU-145 cells, potentially influencing pathways critical to cancer progression [38]. These findings underscore the therapeutic potential of Pd(II) complexes and the need for further research to optimize their efficacy and selectivity.

2.5. DNA Interaction Study

Many metal-based anticancer agents have DNA as a primary potential biological target. Accordingly, it is very important to understand the binding properties of different transition metal ion complexes. Two possible binding modes of the transition metal ion complexes toward DNA are defined as covalent and non-covalent types of interactions. The covalent bindings consider the replacement of the labile ligand of the complex by a nitrogen base of DNA and noncovalent are intercalation, electrostatic, or groove binding. The most employed method for determination of binding mode between complexes and DNA is fluorescence quenching [39]. For this investigation competitive studies with EB and Hoechst, 33258 (Hoe) were made using fluorescence quenching experiments.

Ethidium bromide (EB) is a classical intercalator that gives significant fluorescence emission intensity when it intercalates into the base pairs of DNA. The fluorescence titration of EB-DNA with increasing amounts of 2c is shown in Figure 8. The decrease in the emission intensity of the band at 611 nm suggests that complex 2c can replace EB from EB-DNA and interact with DNA by the interactive mode [40,41].

The Stern-Volmer quenching constants (K_sv) are calculated from the slopes of the plots of I₀/I vs. [4] from Equation 1 [42]

I₀/I=1+K_sv[Q]

(1)

and presented in Table 3. Quenching rate constants, k_q are calculated through the correlation (2), sience its known that the average fluorescence lifetime of the DNA without a quencher (τ₀) is 10⁻⁸ s. [43]

K_sv = k_q× τ₀

(2)

The obtained values for K_sv suggest that 2c can interact with DNA molecules in interactive mode [41].

In order to further elucidate the binding mode of complex 2c with CT-DNA, competitive fluorescence quenching studies were also conducted using Hoechst 33258 (Hoe), a synthetic derivative of N-methylpiperazine known to bind specifically within the minor groove of DNA [7]. Minor groove binding represents a non-covalent interaction typical for small-molecule drugs and bioactive compounds, often associated with sequence-selective recognition [42]. The fluorescence titration of the pre-formed Hoe–DNA complex with increasing concentrations of complex 2c is shown in Figure 9. A gradual decrease in fluorescence intensity suggests that complex 2c can effectively displace Hoe from the minor groove, indicating a possible groove-binding mode of interaction with DNA.

The decrease in fluorescence intensity, as shown in Figure 9, indicates that the palladium complex 2c is capable of displacing Hoe from the Hoe–DNA complex. The Stern–Volmer quenching constant (K_sv) , calculated and presented in Table 3, supports this observation and further confirms the proposed interaction between complex 2c and DNA through minor groove binding.

By comparing the K_sv values obtained from the EB and Hoe fluorescence quenching experiments (Table 3), it can be concluded that the complexes interact with double-stranded DNA through both binding modes – intercalation and minor groove binding. However, the higher Stern–Volmer constant observed for the Hoe displacement assay compared to the EB assay suggests that complex 2c exhibits a stronger preference for minor groove binding over interactive interaction.

2.6. Molecular Docking Study

The inhibition of the AR plays a crucial role in the treatment of androgen-dependent diseases, particularly prostate cancer. Targeting AR with small molecules can modulate its activity, thereby disrupting androgen signaling pathways critical for tumor growth and progression. In this context, our molecular docking study aimed to evaluate the binding affinity of Pd-based complexes as potential AR inhibitors. As shown in Table 4, molecular docking results indicate that the 2c complex exhibits the strongest interaction with the receptor, as evidenced by its lowest binding free energy value. Its inhibition constant, the smallest among the tested compounds, suggests a high potential for enzyme inhibition, making it the most promising candidate in this study. The 2b complex also demonstrates significant inhibitory activity, albeit slightly weaker than 2c, yet it still forms stable interactions with the receptor, highlighting its relevance as a potential inhibitor. In contrast, the 2a complex shows the highest binding free energy value and the largest inhibition constant, indicating a weaker affinity for the enzyme’s active site. For comparison, bicalutamide (BIC) was used as a reference inhibitor, and the results reveal that 2c exhibits a stronger receptor binding affinity. These findings underscore the potential of Pd complexes as AR receptor inhibitors, with 2c emerging as the most promising candidate for further investigation.

The molecular interactions between Pd derivatives and AR, illustrated in Figure 10, offer valuable insights into the potential inhibitory properties of these compounds. Non-covalent interactions contribute significantly to the stability of the ligand-protein complex by reinforcing the binding affinity within the enzyme’s active site. Hydrophobic interactions predominantly enhance the stability of the complex by facilitating favorable packing and minimizing solvent exposure, while conventional hydrogen bonds provide additional stabilization through directional electrostatic forces, further strengthening the ligand-enzyme interaction.

The hydrophobic segments of palladium derivatives enable interactions with various amino acids, such as A:LEU 704, A:VAL 746, and A:ILE 899, leading to the formation of alkyl/π-alkyl contacts. Additionally, other types of non-covalent interactions, including π-sigma and π-sulfur interactions, play a crucial role in ligand stabilization. π-sigma interactions arise due to the interaction between the electron cloud of aromatic rings and sigma bonds within the molecule [44], as observed in 2c interacting with A:MET 742, A:VAL 746, and A:MET 895. Similarly, 2a and 2b also establish interactions with these amino acids, in addition to A:LEU 873. π-sulfur interactions, which involve interactions between sulfur atoms and the π-electron systems of aromatic rings [45], are characteristic of enzymes containing methionine residues, including A:MET 742, A:MET 745, A:MET 749, A:MET 787, and A:MET 895.

Conventional hydrogen bonds are essential for maintaining the stability of ligand–enzyme complexes [44,46]. These interactions arise between the hydroxyl groups of 2a and 2b derivatives and polar amino acid residues within the enzyme’s active site, such as A:MET 742 and A:MET 787. Moreover, a distinct carbon-hydrogen bond is formed between the 2a derivative and A:ASN 705, further stabilizing the complex and ensuring the proper alignment of the ligand within the catalytic pocket. All these interactions collectively play a crucial role in stabilizing the ligand within the enzyme complex.

The observed interactions with the androgen receptor (AR) suggest that Pd complexes can effectively modulate receptor activity. Overall, these diverse binding modes highlight the potential of palladium derivatives for therapeutic applications, particularly in targeting key regulatory proteins like AR. The molecular docking results indicate that all examined Pd complexes exhibit significant interactions with key regulators of apoptosis – caspase 3 (CASP3), Bcl-2-associated X protein (BAX), and the anti-apoptotic B-cell lymphoma 2 (BCL2) protein. The negative values of the binding free energy (ΔG_bind) suggest the stability of the formed complexes, while variations in the obtained parameters provide insight into the binding preferences of these compounds for pro-apoptotic (CASP3, BAX) and anti-apoptotic (BCL2) proteins.

Analysis of the data in Table 5 highlights 2c as the strongest binding ligand, exhibiting the lowest ΔG_bind values for CASP3 (-8.81 kcal/mol) and BAX (-8.44 kcal/mol). These results suggest that Pd3 preferentially targets pro-apoptotic proteins, which may indicate its potential to induce cell death by activating apoptosis. These findings suggest that 2c exhibits selectivity toward pro-apoptotic proteins, highlighting its potential role as a pro-apoptotic modulator in anticancer strategies. In contrast, 2a and 2b display a more balanced binding affinity between pro-apoptotic and anti-apoptotic proteins, which may imply a different mechanism of action.

Since 2c exhibited the most favorable binding results, it was selected to represent the intermolecular interactions with the active sites of CASP3, BCL2, and BAX, as shown in Figure 11. These interactions involve a combination of hydrophobic, electrostatic, and hydrogen bonding forces. Hydrophobic interactions play a central role in driving the binding process, while electrostatic forces aid in the precise alignment of 2c within the enzyme’s active site. Additionally, hydrogen bonds further stabilize the complex, enhancing both the strength and specificity of the binding. These findings underscore the molecular factors that contribute to 2c’s potential as an effective modulator of apoptotic pathways.

In the context of CASP3 (Figure 11a), 2c engages in a conventional hydrogen bond with A:ARG 207, along with carbon-hydrogen interactions involving A:SER 205 and A:ARG 207. Additionally, alkyl/π-alkyl interactions are observed with A:CYS 163, A:ARG 207, and A:TRP 214. The π-π interactions can adopt different configurations, such as edge-to-face (T-shape) or offset stacking (shifted parallel alignment), depending on the local geometry and chemical environment of the binding site [45,47]. Notably, A:HIS 121 participates in one of these π-π interactions, adopting an edge-to-face (T-shape) configuration. 2c also establishes a π-sulfur interaction with A:CYS 163, further stabilizing its binding. Moreover, electrostatic interactions play a critical role in ligand stabilization, particularly π-anion and π-cation interactions, which arise when the electron cloud of an aromatic ring interacts with an anion or cation [44,48]. In this context, 2c forms a π-cation interaction with A:ARG 207, underscoring the selectivity of its binding. Together, these diverse interaction types contribute to a strong binding affinity, aligning with the observed activation of caspase-3. More broadly, they are essential for the stability and specificity of protein-ligand complexes, as they enhance charge complementarity between the ligand and active-site residues, ultimately strengthening overall binding affinity.

For BCL2 (Figure 11b), 2c establishes a conventional hydrogen bond with A:ASN 102 and a carbon-hydrogen bond with A:ARG 105. Additionally, alkyl/π-alkyl interactions are observed with A:VAL 92, A:LEU 96, A:ARG 105, and A:ALA 108, while π-sulfur hydrophobic interactions occur with A:MET 74. Notably, characteristic π-sigma interactions are also present with A:VAL 92. The inhibition of BCL2 disrupts mitochondrial membrane stability, triggering the release of pro-apoptotic factors such as cytochrome c, a key molecule in the initiation of apoptosis.

In the case of BAX (Figure 11c), 2c establishes a carbon-hydrogen bond with A:ALA 104 and engages in T-shaped π-π interactions with A:PHE 146. Additionally, alkyl/π-alkyl interactions are observed with multiple residues, including A:PHE 97, A:VAL 126, A:LEU 130, and A:ALA 149. These stabilizing interactions enhance BAX activation, promoting pore formation in the mitochondrial membrane, which facilitates the release of apoptotic factors and drives the process of programmed cell death.

The molecular interactions of Pd-based complexes with CASP3, BCL2, and BAX offer valuable insights into their potential role in modulating apoptotic pathways. Through diverse binding modes – including hydrogen bonding, π-π stacking, alkyl/π-alkyl interactions, π-sulfur interactions, and electrostatic forces – these complexes contribute to the stability and specificity of protein-ligand interactions. Collectively, these findings underscore the ability of Pd-based complexes to regulate key apoptotic proteins, highlighting their potential as therapeutic agents for apoptosis-related diseases. The specificity and strength of their interactions with critical residues further reinforce their significance in fine-tuning apoptotic signaling pathways.

3. Materials and Methods

The complete Methodology section is given in Supplementary Material

4. Conclusions

Three novel palladium complexes 2a-c, derived from Schiff base ligands combing salicylic and tyramine/tryptamine or p-hydroxyl benzylamine scaffolds in structure, have been synthesized and characterized. The complexes displayed potent anti-proliferative activity against both human prostate cancer cell lines PC-3 and DU-145 and good selectivity between cancer and normal cells. Complex 2c, combining salicylic and tryptamine structural motifs, has demonstrated even better activity than cisplatin for both tested cell lines, with IC₅₀ values of 7.1 µM (DU-145) and 8.6 µM (PC-3). The apoptotic assay has revealed the potential of complexes for apoptosis induction through the Bcl-2 and caspase family activation pathways. Molecular docking studies revealed that complex 2c exhibits the strongest binding affinity toward the androgen receptor (AR), surpassing the reference inhibitor bicalutamide. Additionally, 2c showed selective interactions with pro-apoptotic proteins CASP3 and BAX, involving stable non-covalent forces such as hydrophobic contacts, π-π stacking, π-sulfur, and hydrogen bonds. These interactions suggest a dual mechanism of action – AR inhibition and pro-apoptotic modulation –supporting 2c’s potential as a multifunctional anticancer agent. Taken together, these findings highlight the therapeutic potential of palladium complex 2c as a promising lead compound for the development of novel, multitarget anticancer agents, particularly in the treatment of prostate cancer.