Molecular and Morphological Evidence of Hepatotoxicity after Silver Nanoparticle Exposure: A Systematic Review, In Silico, and Ultrastructure Investigation
Toxicological Research 2019;35:257−270
Published online July 15, 2019;  https://doi.org/10.5487/TR.2019.35.3.257
© 2019 Korean Society of Toxicology.

Kanidta Sooklert1,†, Asarn Wongjarupong2,†, Sarocha Cherdchom1, Nicha Wongjarupong3, Depicha Jindatip4, Yupa Phungnoi5, Rojrit Rojanathanes6 and Amornpun Sereemaspun1

1Nanomedicine Research Unit, Department of Anatomy, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
2Department of Orthopedics, Queen SavangVadhana Memorial Hospital, Sriracha, Chonburi, Thailand
3Department of Physiology, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
4Department of Anatomy, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
5Department of Biology, Faculty of Science and Technology, Nakhon Ratchasima Rajabhat University, Nakhorn Ratchasima, Thailand
6Department of Chemistry, Faculty of Science, Chulalongkorn University, Bangkok, Thailand
Amornpun Sereemaspun, Nanomedicine Research Unit, Department of Anatomy, Faculty of Medicine, Chulalongkorn University, 1873 Rama 4 Road, Pathumwan, Bangkok 10330 Thailand
E-mail: amornpun.s@chula.ac.th

The first two authors contributed equally to this work.
Received: October 29, 2018; Revised: November 13, 2018; Accepted: January 29, 2019
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract

Silver nanoparticles (AgNPs) have been widely used in a variety of applications in innovative development; consequently, people are more exposed to this particle. Growing concern about toxicity from AgNP exposure has attracted greater attention, while questions about nanosilver-responsive genes and consequences for human health remain unanswered. By considering early detection and prevention of nanotoxicology at the genetic level, this study aimed to identify 1) changes in gene expression levels that could be potential indicators for AgNP toxicity and 2) morphological phenotypes correlating to toxicity of HepG2 cells. To detect possible nanosilver-responsive genes in xenogenic targeted organs, a comprehensive systematic literature review of changes in gene expression in HepG2 cells after AgNP exposure and in silico method, connection up- and down-regulation expression analysis of microarrays (CU-DREAM), were performed. In addition, cells were extracted and processed for transmission electron microscopy to examine ultrastructural alterations. From the Gene Expression Omnibus (GEO) Series database, we selected genes that were up- and down-regulated in AgNPs, but not up- and down-regulated in silver ion exposed cells, as nanosilver-responsive genes. HepG2 cells in the AgNP-treated group showed distinct ultrastructural alterations. Our results suggested potential representative gene data after AgNPs exposure provide insight into assessment and prediction of toxicity from nanosilver exposure.

Keywords : Silver nanoparticles, Ultrastructural alterations, Silver-nanoparticle responsive gene, Systematic review
INTRODUCTION

Nanotechnology has become unavoidable in modern science, as it solves many human-related problems (1). In our daily lives, nanoparticles with diameters of various sizes between 1–100 nm appear in many areas (2). Silver nanoparticles (AgNPs), a commonly utilized nanoparticle, exhibit extraordinary physical, chemical, and biological properties. Indeed, AgNPs have been widely used across medical fields for their antimicrobial, antiviral, and antifungal properties (36). Thus, it is unsurprising that nanosilver toxicity is particularly concerning because of its potential cytotoxic effects.

Researchers have recently become interested in the genotoxicity associated with AgNP exposure. Routes by which the human body could be exposed to AgNPs include direct contact, inhalation, ingestion, and intraperitoneal or intravenous injection, with nanoparticles subsequently accumulating in a diverse group of vital organs including lung, kidney, spleen, brain, and liver (7). In animal studies, the highest concentration of AgNPs was observed in liver (810). According to the limitations of using animal models, a human in vitro system become alternative approaches for cytotoxic research. Human hepatoma cell line (HepG2 cell) has been widely used as a model of hepatocytes. These cells are epithelial in morphology and display many of the genotypic features of normal liver cells. Moreover, HepG2 cells have been used as a cytotoxic screening cell line for several drugs and chemicals, and to evaluate toxicity after AgNP exposure (11,12).

It is generally assumed that the toxicological effects of AgNPs mainly arise from induction of cellular oxidative stress (13). It is also accepted that the genotoxic potential of AgNPs derives from the toxicological consequences of oxidative damage induced by AgNP exposure (14). Mechanisms underlying genotoxic effects of nanoparticles have been studied using cell lines in most cases. However, potential nanosilver-responsive genes have not been established. In addition, correlation between specific morphological changes and genetic pathway responses remain unclear. Thus, this study aimed to demonstrate ultrastructural alterations in HepG2 cells after AgNP exposure, with particular interest in examining correlations between ultrastructural alterations and AgNP-responsive genes. We first conducted an in silico screen and systematic review to detect possible nanosilver-responsive genes that could be potential indicators for AgNP toxicity. The utility of combining transmission electron microscopy (TEM) and gene detection to evaluate toxicity after AgNP exposure was also evaluated. Our results warrant further studies investigating the correlation of molecular targets and morphological alterations.

MATERIALS AND METHODS

Systematic review of gene expression

A systematic review was conducted to identify relevant literature analyzing cytotoxic effects of AgNPs in HepG2 cells in terms of altered gene expression. We performed a literature search of three electronic databases: PubMed, Scopus, and ISI Web of Science from all recorded data up to January 2018. PubMed search terms were (“silver” or “silver nanoparticle” or “AgNP”) AND (“gene expression” OR genotoxic*) AND (liver OR hepatocyte* OR hepg2). Similar search terms and search strategy were used for both Scopus and Web of Science. Two authors independently screened titles and abstracts for studies of gene expression after AgNP exposure in human cells and obtained full texts. Inclusion criteria were (i) experimental study with control, (ii) use of HepG2 cells, (iii) exposure to AgNPs of any size, (iv) and fold-change gene expression (either up or down) as an outcome measure. Selected articles were independently reviewed by two additional authors, and data describing gene expression responses to AgNP administration were extracted. In cases where more than one outcome was obtained for a gene in the study, we extracted the result of the experiment with the most similar exposure to our morphological study.

Microarray data and data analysis

Gene expression microarray data for AgNP- and silver ion (Ag+)-exposed HepG2 cells were obtained from the GEO (https://www.ncbi.nlm.nih.gov/geo/) database. The search term was “nano, nanoparticle, hepg2, and hepatocyte”. Original studies comparing gene expression between AgNPs and Ag+ were included. AgNP-specific genes were defined as (1) causing up-regulation in AgNP-exposed cells and not up-regulation in cells exposed to Ag+ and (2) causing down-regulation in AgNP-exposed cells and not down-regulation in cells exposed to Ag+. The eligible GEO series accession number was GSE14452. Raw data processed by a two-tailed Student’s t-test were used to compare mean expression levels and identify AgNP-specific genes. Statistical significance was set at p < 0.001. All statistical analysis steps were processed by Connection Up- and Down-Regulation Expression Analysis of Microarrays (CU-DREAM) software. Overall overlaps in gene expression, defined as AgNP-specific genes, were categorized into two main categories by their subcellular localization and molecular function.

Characterization of AgNPs

Silver nanoparticles of particle size 10 nm in aqueous buffer, containing sodium citrate as the stabilizer, were purchased from Sigma-Aldrich (St. Louis, MO, USA). The particle size and shape of AgNPs were determined using transmission electron microscope (TEM) equipped with a charge-coupled device (CCD) camera (Hitachi High-Technologies Corporation, Tokyo, Japan). To characterize the surface charge and absorption wavelength, the commercial AgNPs solution was analyzed by Zetasizer (Malvern Instruments, Malvern, UK) and ultraviolet (UV) spectrophotometer (Beckman Coulter, Brea, CA, USA) respectively.

Cytotoxicity of AgNPs and their released ion

The cytotoxicity study of AgNPs was conducted in 96-well plates. A total of 5 × 103 cells in 45 μL DMEM were seeded into each well and incubated overnight at 37°C with 5% CO2. AgNPs solution at various concentrations (1, 3, 5, 7, 10, 15, 20, 25, 30, 35, 40, 50, 75, and 100 μg/mL) was then administered to HepG2 cells for 24 hr in order to determine the appropriate dose for induction of cytotoxicity. Prior to AgNPs treatment of HepG2 cells, each dilution of AgNPs was dispersed in Dulbecco’s modified Eagle medium (DMEM) by pipetting. The treatment concentration of AgNPs was determined by the half maximal inhibitory concentration (IC50) value. To study the cytotoxic effect of AgNPs and their released ion, an Agion solution was prepared by incubating the AgNPs solution at IC50 dose in DMEM for 24 hr at 37°C, and then centrifuging at 20,000 rpm for 2 hr. Supernatant was collected in order to analyse the released Ag-ion content using inductively coupled plasma optical emission spectrometry (ICP-OES) (PerkinElmer, Waltham, MA, USA). PrestoBlueTM reagent (Invitrogen, Carlsbad, CA, USA), a cell-permeable compound that becomes highly fluorescent through reduction by dehydrogenase, was then added to each well and the plates were incubated at 37°C for 30 min. A quantitative analysis of cell viability was determined by measurement of the fluorescence intensity using a fluorescence multi-well plate reader (Varioskan Flash, Thermo Fisher Scientific, Waltham, MA, USA) at 530 nm excitation and 590 nm emission.

Evaluation of ultrastructural changes

Ultrastructural alterations in HepG2 cells exposed to AgNPs were visualized by TEM. For preparation of samples, HepG2 cells were seeded in 6-well plates at a density of 1 × 106 cells/well and incubated at 37°C and 5% CO2 for 24 hr. Cells were treated with 0.5, 2, or 6 μg/mL AgNPs (Sigma-Aldrich) and incubated for an additional 24 hr. Cells treated with silver nanoparticles were harvested by trypsinization and centrifugation. Samples of HepG2 cells were fixed in 2% glutaraldehyde for 2 hr at room temperature and then rinsed with phosphate buffer. Next, samples were postfixed in 2% osmium tetraoxide solution and subsequently dehydrated with ethanol. Samples were then embedded in epoxy resin and blocks were polymerized at 60°C overnight. After polymerization, thick sections of samples were selected for staining with toluidine blue and examined under a light microscope. Ultrathin sections of these samples were mounted on copper grids and counterstained with 2% uranyl acetate and lead citrate. Stained sections were examined using a Hitachi transmission electron microscope H-7650 at an operating voltage of 100 kV.

Analysis of death pattern

The patterns of cell death; apoptosis and necrosis were examined by flow cytometry (AMNIS, EDM Millipore, Darmstadt, Germany). HepG2 cells were plated in 6-well plates at a density of 1 × 106 cells/well and incubated at 37°C and 5% CO2 for 24 hr. The cells were treated under the same conditions as those for the ultrastructural alteration study. The control and treated cells were harvested by trypsin and washed twice with cold phosphate-buffered saline (PBS) before staining with fluorescein isothiocyanate (FITC)-conjugated annexin V and propidium iodide (PI) (BD Biosciences, Franklin Lakes, NJ, USA) according to the manufacturer’s instructions. After staining, the cells were analyzed within 1 hr by the ImageStream flow cytometer using emission filters for FITC at 532/55 nm and PI at 610/30 nm. The percentage of cells undergoing apoptosis or necrosis was analyzed by IDEAS®ImageStream software (AMNIS). To study another one of the major death patterns, a prominent marker microtubule-associated protein light chain 3 (LC3) protein expressions was used to indicate autophagy by western blotting. Equal amounts of total extracted protein were loaded onto a 10% polyacrylamide gel and separated by electrophoresis. The separated proteins were transferred to a polyvinylidene difluoride (PVDF) membrane. Afterwards, the membrane was incubated with a 1:1000 dilution of anti-LC3 antibody (rabbit polyclonal anti-LC3; Sigma-Aldrich, St. Louis, MO, USA) overnight at 4°C. After three times rinsed with TBS-T buffer, the membrane was incubated for 2 hr at room temperature with horseradish peroxidase conjugated secondary antibodies, dilution 1:5000 (Kirkegaard & Perry Laboratories, Gaithersburg, MD, USA). The immunoreactive bands were detected with an enhanced chemiluminescence assay ECL (GE Healthcare Life Sciences, USA) and visualized using Image Studio Digits for C-DiGit® Blot Scanner (LI-COR Biosciences, Lincoln, NE, USA). Equal loading of proteins was confirmed by measuring the internal control β-actin.

RESULTS

Integration of systematic review

We identified 452 articles from our electronic search accordance with diagram in Fig. 1. From abstract screening, there were nine potential studies, seven of which involved HepG2 cells. All seven studies met our inclusion criteria and were included in this systematic review (Table 1). The size of AgNPs used varied from 10–200 nm, and the dosage range varied from 0.02–100 μg/mL. Durations of exposure included 2, 4, 6, and/or 24 hr. Overall, 36 genes were identified to be significantly up-regulated after exposure to AgNPs, while eight were significantly down-regulated. There were six genes that were identified in more than one study. Three studies demonstrated significant up-regulation of FOS and JUN genes, whereas two studies demonstrated significant up-regulation of EGR1, CXCL8, HSPB1, and MT2A genes. Fold-increases in gene expression varied among studies. Notably, genes involved in cell proliferation, such as FOS, JUN, and EGR1, exhibited significantly increased fold-changes with high-dosage exposures. Gene localization and related pathways (primarily transcription factor, metal ion binding, and chemokine/cytokine activity) are described in Table 2.

Genes up- or down-regulated in response to AgNP exposure

To identify candidate target genes that are up- or down-regulated only in AgNP-exposed cells, genes expression in cells exposed to AgNPs or Ag+ were compared. The number of genes in the AgNP-exposed group that significantly intersected with the Ag+-exposed group is shown in Table 3. The 18 genes most significantly up-regulated and 6 genes down-regulated only in the AgNP-exposed group were identified and sub classified. A list of selected genes and their molecular function is shown in Table 4. The molecular functions of selected genes were annotated using complementary information available from Gene-NCBI and UNIPROT databases.

Characteristics of AgNPs

To enable a proper comparison with the literature review, the characteristics of the commercial AgNPs were analyzed and shown in Fig. 2. Our results confirmed that the shapes of the nanoparticles are near-spherical and the size of the nanoparticles is uniform with a mean diameter of 10 nm (Fig. 2B). UV-visible absorption peak of AgNPs was 403.3 nm (Fig. 2A) and the average zeta potential was −20.7 mV, suggesting the AgNPs are relatively stable and monodispersed in culture medium.

Cytotoxic effect of AgNPs

The cytotoxic effect of AgNPs was assessed by a PrestoBlueTM cell viability assay and the IC50 value was calculated by using commercialized Graphpad Prism® software (GraphPad Software, Inc., San Diego, CA, USA) for the subsequent experiments. As shown in Fig. 3A, AgNPs exert cytotoxic effects on HepG2 cells in a dose-dependent manner with IC50 value of 6.28 μg/mL. To determine whether AgNP-induced cytotoxicity was depended primarily on the dose of AgNPs, we investigated the Ag-ion content released from AgNPs at IC50 dose. As measured by ICP-OES, the Ag-ion content released from AgNPs was 0.037 + 0.0008 μg/mL. This released Ag-ion content accounts for 0.61 percent of the total AgNPs solution (6 μg/mL). Compared with the control group, Ag ion-treated cells showed no significantly changed in cell viability. The cytotoxicity was seen only in AgNPs treated cell (Fig. 3B).

Ultrastructural alterations in HepG2 cells

In an attempt to evaluate possible correlations between changes in expression levels of AgNP-responsive genes and alterations in the ultrastructural morphology of AgNP-treated cells, we used TEM to observe morphological changes in AgNP-exposed cells. AgNP-exposed cells exhibited characteristics of different death patterns. Non-cytotoxic dosages of AgNPs (0.5 and 2 μg/mL) were found to induce assembly of degraded organelles in the cytoplasm. HepG2 cells exposed to AgNP at non-toxic concentrations exhibited numerous initial autophagic vacuoles and multiple vacuoles containing degradative contents, called autolysosomes (Fig. 4A, 4B). Furthermore, HepG2 cells exposed to AgNP at a higher concentration (6 μg/mL) demonstrated a distinct morphology indicating necrotic cell death. At cytotoxic dosages, AgNPs were found to influence cell membrane rupture and damage to organelles (Fig. 4C).

Apart from changes in ultrastructural morphology to indicate the pattern of death, TEM images demonstrated that AgNP-exposed cells exhibited changes in cytoplasmic organelles compared with unexposed cells, as shown in Fig. 5. The control group showed normal ultrastructural morphology (Fig. 5A), while the AgNP-treated group showed distinct morphological changes (Fig. 5B–5F) indicating unhealthy cells. As shown in Fig. 5A, untreated cells exhibited a round nucleus with homogeneous chromatin, normal mitochondria, rough endoplasmic reticulum (exhibiting a distinct ultrastructural appearance), and intact nuclear membrane. Microscopic images from the AgNP-treated group showed distortion of nuclear membranes, formation of blebbed nuclei, and accumulation of autophagic vacuoles containing degradative organelles (Fig. 5B, 5D). Ultrastructural morphology of AgNP-exposed cells demonstrated increased numbers of mitochondria and cytoplasmic vacuoles containing silver nanoparticles. In addition, numerous swollen lipid droplets were observed in the cytoplasm (Fig. 5C). Impairment of other cytoplasmic organelles was also apparent, including swollen mitochondria and disappearance of mitochondrial inner membranes (Fig. 5E), and dilated Golgi saccules (Fig. 5F).

Assessment of the cell death pattern

To confirm the dose involvement of the different pattern of cell death upon our TEM results, double staining FITC-conjugated annexin V and PI, and immunoblotting of LC3 were employed to examine the three major patterns of cell death; apoptosis, necrosis and autophagy. HepG2 cells were treated with varying concentrations of AgNPs to demonstrate the dose-dependent effect on the death pattern. Flow cytometry analysis showed the 99.68% of untreated control cells were viable. At the sub-cytotoxic dosages of 0.5 and 2 μg/mL AgNPs, the majority of cells had a higher percentage of apoptotic cells (11.1% and 21.57%, respectively) compared with control while the cells treated with the cytotoxic dose of AgNPs (6 μg/mL) was preferable to be necrotic cells (42.4%). Results were representative of n = 3 (Fig. 6A, 6B). As showed in Fig. 7A, 7B, the subcytotoxic dosages of AgNPs showed upregulation of LC3 protein, an established molecular marker for detect autophagosome formation, expression while 6 μg/mL AgNPs induce expression of LC3 protein were not detected.

DISCUSSION

AgNPs are an influential material with increasing applications in biomedical science and commercial products. Simultaneously, concerns about their use have recently been increasing because of their cellular toxicity (15). However, exactly how AgNPs affect cytotoxicity remains unclear. Several hypotheses suggest exposure to AgNPs induces gene alterations in mammalian cells, which can affect cell functions and cause morphological alterations (16,17). The results of the present study indicated target genes responsible for the effects of exposure to AgNPs. Most genes were located in the nucleus, where they appear to function as transcription factors. In addition, this report provides electron microscopic evidence of AgNP-induced toxicity in hepatic cells. Indeed, abundant vacuoles and morphological changes in subcellular organelles such as nuclear membranes, mitochondria, and golgi apparatus were only observed in AgNP-exposed cells.

Based on our systematic literature review and computational analyses, 44 genes and 24 genes were found to be up- or down-regulated, respectively, in response to AgNP exposure. These 68 gene candidates were categorized by their molecular function. Among them, the molecular function of most genes affected by AgNPs exposure was potentially relevant to transcriptional control. The genes in the CCAAT enhancer binding protein (CEBP) family were found in both methods, CCAAT enhancer binding protein beta (CEBPB) was discovered from the systematic literature review and CCAAT enhancer binding protein alpha (CEBPA) was discovered from the in silico analysis. The CEBP family is an important transcription factor regulating the expression of genes involved in immune and inflammatory responses. These two members of the CEBP family were reported to be the key transcriptional control of liver regeneration (18). In hepatocyte, CEBPA is highly expressed and play critical roles in regulating many metabolic liver genes, while CEBPB is up-regulated during liver regeneration and play crucial roles in the development of liver or acute inflammatory response (19). Interference of the expression level of these main regulators of liver function by AgNPs possibly facilitates the malfunction of hepatocyte. To our knowledge, the genes in CEBP family are of great concern when studying the cytotoxic mechanism of AgNPs.

Although another candidate gene obtained from the systematic literature review and computational analyses were dissimilar, their molecular functions were found to be related. We hypothesized the differences sets of genes found in both methods were the consequence of difference in the sizes, concentration, and stabilizer of AgNPs. It should be noted that target genes in liver cells are involved in gene transcription control, metal ion binding, ATP binding, protein binding, and antioxidant activity, among other functions.

The main finding in our systematic review and microarray analysis indicated that genes changing in response to AgNP exposure are primarily responsible for oxidative stress, inflammation, and cell death. In this study, we found that the proto-oncogenes FOS and JUN, which are known to play important roles in both cell survival and the signaling pathway involved in hepatotoxicity, were highly up-regulated in the presence of AgNPs (2024). In addition, heat shock protein family members HSPB1, HSPA4L, and HSPH1 were also significantly up-regulated (22,25). Heat shock protein is one of the molecules induced to protect cells from oxidative stress. A previous study reported that AgNPs induced oxidative stress, and consequently increased expression of heat shock protein and heme oxygenase (HMOX1) in both liver and lung cells (26). Moreover, AgNPs induced reactive oxygen species (ROS) generation and increased the expression level of HIF1A (27). In addition, augmentation of HIF1A expression reportedly affected the declination of ROS levels by changing oxidative phosphorylation to anaerobic glycolysis in lung cancer cells (28). Genes belonging to the metallothionein family, such as MT1M, MT1H, MT2A, MT1B, MT1X, MT1G, and MT1F, also responded to AgNPs (25,29). Metallothioneins, which have a high affinity for heavy metals, provide a cytoprotective effect against oxidative stress from metal particles including AgNPs (3032). Additionally, the antioxidant-related genes superoxide dismutase (SOD1 and SOD3) and catalase (CAT) were also up-regulated by exposure to AgNPs (33). As a summary of our literature review, it was concluded that AgNPs elicit cytotoxicity through ROS generation and oxidative stress. Thus, our results may provide a possible biomarker for predicting AgNP cytotoxicity. In our in silico analysis, we also identified 24 interesting candidate genes as possible target of AgNPs induce hepatocellular toxicity. We found SOX15 (SRY-box 15) and TLL1 (tolloid like 1) were the most strongly up- and down-regulated by AgNPs respectively. SOX15, the nuclear gene encodes a member of the SOX family, which acts as transcription activator involved in the embryonic development regulation and in the cell fate determination. In testicular embryonal cell, overexpression of SOX15 was reported to cause an inhibition of cell proliferation and G0/G1 phase cell cycle arrest (34). TLL1, the most noticeable down-regulated gene among our results, is the gene belongs to tolloid-like proteins family which is necessary for various developmental events. The down regulation of this gene by prenatal chronic stress was hypothesized that there would be associated with the process of neurogenesis (35). Based on the present results, AgNPs may exert cytotoxic effects through SOX15 up regulation or TLL1 down regulation in hepatic cell. Further research is needed to explore the mechanism of these genes in AgNP-induced hepatotoxicity.

Evaluation of the potential risks of AgNPs from different aspects will help the design of safer and more effective antimicrobial products. The antimicrobial effects of AgNPs are based on oxidative stress generation, which consequently results in cytotoxicity, genotoxicity, immunological responses, and even cell death (36). In this study, we not only evaluated genomic alterations, but also the ultrastructural morphological status of AgNP-exposed hepatic cells to understand the mechanisms of nanocytotoxicity. Our results revealed that potential target genes are mostly located within the nucleus and cytoplasm. Moreover, significant morphological changes were observed in the nuclei and cytoplasm of AgNP-exposed cells. Changes in gene expression in response to AgNP exposure involve both cellular morphological changes and cell death.

We observed autophagic vacuoles, a characteristic morphology of autophagic cell death, at low AgNP concentrations. However, at high concentrations, we observed severe membrane rupture typical of necrotic cell death. Autophagy is considered to be a survival mechanism that regulates the extent of apoptosis and necrosis (37). Deregulated autophagy after AgNP treatment may lead to increased cell death either independently or synergistically with apoptosis or necrosis (3840). These morphological observations are in accordance with our in vitro assays that showed AgNPs triggered different modes of cell death in a dose dependent manner. Taken together with our results, the dose-response relationship between AgNPs and types of cell death is a necessary for understanding the toxic mechanism of AgNPs.

We demonstrated a possible correlation between AgNP-responsive genes and morphological alterations. These findings provide evidence for the prediction of possible target molecules of AgNP-induced cytotoxicity. However, this study was unable to show a direct correlation between changes in biological markers and alterations in ultrastructural morphology of liver cells after exposure to AgNPs. Further studies are needed to determine the exact target genes of AgNPs and their correlation to morphological alterations of cells. Only then will we understand the precise mechanism and major determinant factors by which AgNPs are toxic to cells.

In summary, concerns about the toxicity of AgNPs require estimation of its safety, and considerable studies have been carried out to examine biomolecular and morphological alterations induced by this nanoparticle. To the best of our knowledge, we are the first to estimate candidate AgNP-targeted genes and correlate these findings with biomolecular responses and ultrastructural alterations in HepG2 cells after AgNP exposure. This research provides comprehensive and systematic analysis for identify unclear knowledge of cellular response to AgNP cytotoxicity.

ACKNOWLEDGMENTS

The authors gratefully acknowledge the Grant to support a research group in the Ratchadaphiseksomphot Endowment Fund, Chulalongkorn University. Authors would like to thank Assistant Professor Dr. Supang Maneesri-Le Grand from Department of Pathology, Faculty of Medicine, Chulalongkorn University, Thailand, for her useful critiques and valuable technical support on this study. KS would like to acknowledge her postdoctoral fellowship supported by the Ratchadapiseksompote Endowment Fund, Chulalongkorn University. SC was supported by the 100th Anniversary Fund Chulalongkorn University for Doctoral Scholarship. This research has been supported by the Chula Unisearch, Chulalongkorn University (NRU59-017-HR), and partially supported by the Ratchadaphiseksomphot Endowment Fund of Chulalongkorn University (RES560530230-AM) and the Ratchadapiseksomphot Fund, Faculty of Medicine, Chulalongkorn University (RA58/053).

LIST OF ABBREVIATIONS
Ag+: Silver ion
AgNPs: Silver nanoparticles
CCD: Charge-coupled device
CEBP: CCAAT enhancer binding protein
CEBPA: CCAAT enhancer binding protein alpha
CEBPB: CCAAT enhancer binding protein beta
CU-DREAM: Connection up- and down-regulation expression analysis of microarrays
FITC: Fluorescein isothiocyanate
GEO: Gene Expression Omnibus
IC50: Half maximal inhibitory concentration
ICP-OES: Inductively coupled plasma optical emission spectrometry
LC3: Microtubule-associated protein light chain 3
PBS: Phosphate-buffered saline
PI: Propidium iodide
PVDF: Polyvinylidene difluoride
TEM: Transmission electron microscopy
UV: Ultraviolet.
Figures
Fig. 1. Flow diagram illustrating systematic literature selection process in accordance with PRISMA guideline.
Fig. 2. Characterization of AgNPs. (A) UV-Vis spectrum of AgNPs. (B) Representative TEM image of AgNPs (scale bar: 50 nm).
Fig. 3. Effect of AgNPs on the viability of HepG2 cells at 24 hr. (A) A dose dependent decrease in percentage of viable cells following AgNPs exposure. (B) Percentage of viable cells following AgNPs and Ag ions exposure. Data are the average of five replicate samples (*p< 0.01 compared with the Ag ions-treated group, **p< 0.001 compared with the control group).
Fig. 4. Morphological death patterns of cells treated with AgNPs. Transmission electron image of HepG2 cells incubated for 24 hr with (A) 0.5 μg/mL AgNPs, (B) 2 μg/mL AgNPs or (C) 6 μg/mL AgNPs.
Fig. 5. AgNPs induced the changes in the ultrastructural morphology of cytoplasmic organelles. Cells were treated with (A) control, (B–F) AgNPs (6 μg/mL) for 24 hr and processed for TEM. Notes: Representative TEM micrograph of AgNP-treated cells showed (B) nuclear membrane distortion (black arrow) and numerous initial autophagic vacuoles (Avi), (C) internalization of aggregated-AgNPs in cytosolic vesicles (black stars) and swollen lipid droplets (white stars), (D) blebbed nuclei (black arrow) and accumulation of both initial autophagic vacuoles (inset) and degradative autophagic vacuoles (Avd), (E) swollen mitochondria and mitochondrial cristae disappear (double arrows), and (F) dilation of Golgi saccules (white arrow).
Fig. 6. Death pattern analysis of cells treated with AgNPs. (A) Representative histograms from flow cytometry of HepG2 cells treated with different concentrations of AgNPs. (B) Distribution of viable, apoptotic, and necrotic cells treated with different concentrations of AgNPs as measured by annexin V and PI staining.
Fig. 7. AgNPs induce autophagy in HepG2 cells. (A) Western blot analysis of autophagy related proteins (LC3) in HepG2 cells treated with a series of concentrations of AgNPs. (B) Quantification of the LC3-II/β-actin ratio (**p< 0.001 and ***p< 0.0001 compared with the control group).
TABLES

Table 1

A systematic review of genotoxicity after AgNP exposure in HepG2 cells

Study AgNPs exposure in HepG2 cell line Gene name Gene symbol Major outcome (fold change)

Particle size (nm) Doses (μg/mL) Exposure time (hr)
Brzoska et al. (22) 20, 200 100 2, 6, 24 Fos proto-oncogene FOSa Up-regulation (234.26)
Early growth response 1 EGR1b Up-regulation (27.78)
Selectin P ligand SELPLG Up-regulation (20.55)
Jun proto-oncogene JUNc Up-regulation (19.95)
C-X-C motif chemokine ligand 8 CXCL8 (IL8)d Up-regulation (18.05)
Matrix metallopeptidase 10 MMP10 Up-regulation (15.70)
WNT1 inducible signaling pathway protein 1 WISP1 Up-regulation (10.95)
Growth arrest and DNA damage inducible alpha GADD45A Up-regulation (7.59)
Heat shock protein family B (small) member 1 HSPB1e Up-regulation (5.05)
Interleukin 4 IL4 Up-regulation (4.69)
CCAAT/enhancer binding protein beta CEBPB Up-regulation (3.28)
Vascular endothelial growth factor A VEGFA Up-regulation (2.86)
C-C motif chemokine ligand 2 CCL2 Up-regulation (2.34)
BCL2 related protein A1 BCL2A1 Up-regulation (2.16)
Hexokinase 2 HK2 Up-regulation (1.95)
Cyclin dependent kinase inhibitor 1B CDKN1B Down-regulation (2.16)
Transferrin receptor TFRC Down-regulation (2.09)
Growth regulation by estrogen in breast cancer 1 GREB1 Down-regulation (1.61)
Forkhead box A2 FOXA2 Down-regulation (1.52)

Sahu et al. (29) 20, 50 2.5 4, 24 Metallothionein 1M MT1M Up-regulation (5.71)
Metallothionein 1B MT1B Up-regulation (4.04)
Metallothionein 1G MT1G Up-regulation (3.36)
Metallothionein 2A MT2Af Up-regulation (1.91)
Metallothionein 1F MT1F Up-regulation (1.89)
SRY-box 4 SOX4 Up-regulation (1.80)

Stepkowski et al. (24) 20, 200 100 6 Fos proto-oncogene FOSa Up-regulation (247.39)
Early growth response 1 EGR1b Up-regulation (26.38)
Jun proto-oncogene JUNc Up-regulation (24.34)
Heme oxygenase 1 HMOX1 Up-regulation (24.17)
C-X-C motif chemokine ligand 8 CXCL8 (IL8)d Up-regulation (18.08)
Interleukin 10 IL10 Up-regulation (5.59)
Interferon beta 1 IFNB1 Up-regulation (2.56)
Mitogen-activated protein kinase kinase kinase 1 MAP3K1 Down-regulation (1.96)
Tumor necrosis factor superfamily member 10 TNFSF10 Down-regulation (1.77)
Receptor interacting serine/threonine kinase 1 RIPK1 Down-regulation (1.74)
Interleukin 1 receptor type 1 IL1R1 Down-regulation (1.63)

Jiao et al. (23) 10, 100 0.2, 0.5, 1, 2 24 Jun proto-oncogene JUNc Up-regulation (2.3)
Fos proto-oncogene FOSa Up-regulation (2.1)

Garcia-Reyero et al. (27) 30 0.002–0.2 24 Superoxide dismutase 3 SOD3 Up-regulation (13.18)
Signal transducer and activator of transcription 1 STAT1 Up-regulation (3.55)
Tumor protein p53 TP53 Up-regulation (2.93)
Signal transducer and activator of transcription 3 FLOT1 Up-regulation (2.57)
Flotillin 1 STAT3 Up-regulation (2.25)
Hypoxia inducible factor 1 alpha subunit HIF1A Up-regulation (1.94)

Kim et al. (33) 10 0.2 24 Catalase CAT Up-regulation (11.00)
Superoxide dismutase 1 SOD1 Up-regulation (4.50)

Kawata et al. (25) 10 0.1–3 24 Metallothionein 1H MT1H Up-regulation (4.50)
Metallothionein 2A MT2Af Up-regulation (4.10)
Metallothionein 1X MT1X Up-regulation (3.4)
Heat shock protein family A (Hsp70) member 4 like HSPA4L Up-regulation (2.2)
Heat shock protein family B (small) member 1 HSPB1e Up-regulation (2.1)
Heat shock protein family H (Hsp110) member 1 HSPH1 Up-regulation (2.1)

a,b,c,d,e,f

Indicates a gene which is present in multiple citations.


Table 2

Characteristics of selected genes with significantly increased/decreased expression after AgNP exposure, based on a systematic review of HepG2 cells

Subcellular location Molecular function Gene
Nucleus Transcription factor FOS, JUN, STAT1, CEBPB, STAT3, HIF1A, SOX4, FOXA2*, EGR1
Metal ion binding MT1M, MT1H, MT2A, MT1B, MT1X, MT1G, MT1F
ATP binding, Protein modification TP53, HSPB1, HSPA4L, HSPH1
Protein binding, Protein phosphate binding GADD45A, CDKN1B*
Mitochondria Hexokinase activity, ATP binding HK2
Metal ion binding (Intracellular antioxidant activity) SOD1
Protein heterodimerization/homodimerization activity BCL2A1
Endoplasmic reticulum Heme oxygenase activity HMOX1
Peroxisome Catalase activity, Antioxidant activity CAT
Cytoplasm Protein kinase binding MAP3K1*
Cell membrane Protein heterodimerization activity FLOT1
Transferrin transmembrane transporter activity TFRC*
Metal ion biding, Receptor binding TNFSF10*
Receptor binding SELPLG
Death receptor binding, Protein serine/threonine kinase activity RIPK1*
Interleukin-1 receptor activity IL1R1*
Hormone-responsive tissues and cancer GREB1*
Extracellular space Metal ion binding (Extracellular antioxidant activity) SOD3
Metalloendopeptidase activity, Metal ion binding MMP10
Chemokine and cytokine activity CXCL8, IL10, IL4, IFNB1, CCL2
Vascular endothelial growth factor receptor binding VEGFA
Heparin binding, Integrin binding WISP1

*

Indicates when a gene is down regulated by AgNPs.


Table 3

Connection up- and down-regulation expression analysis of microarray (CU-DREAM) results. Gene intersections between HepG2 cells exposed to AgNPs with cysteine, astrong ionic silver ligand, and Ag ions

Ag ion AgNPs
Up (p < 0.001) Not up
Up (p < 0.001) 0 7
Not up 18 8,187
P-value: 9.01E-01
Ag ion AgNPs
Down (p < 0.001) Not down
Down (p < 0.001) 7 16
Not down 6 8,183
P-value: 7.35E-293

Table 4

Characteristics of genes specifically up- and down-regulated by AgNP, but not Ag ions

Subcellular location Molecular function Gene
Nucleus Transcription factor CEBPA, MYBL2, CD3EAP, SUV39H1, SOX15, MAFF*
Nucleus, Cytoplasm ATP binding CKB, RECQL4
GTPase activator activity RGS10
Cytoplasm ATP binding XYLB
ATP binding, Protein kinase activity MAP2K2
Protein homodimerization activity FASN
Cadherin binding TAGLN2
Protease binding, Structural molecule activity CSTA*
Cell membrane ATPase coupled ion transmembrane transporter activity ATP6V0E2
ATPase activator activity RAB3A
Nucleoside transmembrane transporter activity SLC29A1
Extracellular space Erythropoietin receptor binding, Hormone activity EPO
Growth factor and cytokine activity BMP6
Enzyme inhibitor activity SCG5*
Chemokine activity CCL20*
Calcium ion binding, Metalloendopeptidase activity TLL1*
Carbohydrate binding, Growth factor activity REG1A*
Cytoskeleton Cadherin binding, Structural molecule activity EVPL

*

Indicates when a gene is down regulated by AgNPs.


References
  1. Dilnawaz, F, Acharya, S, and Sahoo, SK (2018). Recent trends of nanomedicinal approaches in clinics. Int J Pharm. 538, 263-278.
    Pubmed CrossRef
  2. Griffin, S, Masood, MI, Nasim, MJ, Sarfraz, M, Ebokaiwe, AP, Schafer, KH, Keck, CM, and Jacob, C (2017). Natural nanoparticles: a particular matter inspired by nature. Antioxidants (Basel). 7, E3.
    Pubmed KoreaMed CrossRef
  3. Noronha, VT, Paula, AJ, Duran, G, Galembeck, A, Cogo-Muller, K, Franz-Montan, M, and Duran, N (2017). Silver nanoparticles in dentistry. Dent Mater. 33, 1110-1126.
    Pubmed CrossRef
  4. Pugazhendhi, A, Prabakar, D, Jacob, JM, Karuppusamy, I, and Saratale, RG (2017). Synthesis and characterization of silver nanoparticles using Gelidium amansii and its antimicrobial property against various pathogenic bacteria. Microb Pathog. 114, 41-45.
    Pubmed CrossRef
  5. Rai, M, Ingle, AP, Paralikar, P, Gupta, I, Medici, S, and Santos, CA (2017). Recent advances in use of silver nanoparticles as antimalarial agents. Int J Pharm. 526, 254-270.
    Pubmed CrossRef
  6. Vimbela, GV, Ngo, SM, Fraze, C, Yang, L, and Stout, DA (2017). Antibacterial properties and toxicity from metallic nanomaterials. Int J Nanomedicine. 12, 3941-3965.
    Pubmed CrossRef
  7. Fewtrell, L, Majuru, B, and Hunter, PR (2017). A re-assessment of the safety of silver in household water treatment: rapid systematic review of mammalian in vivo genotoxicity studies. Environ Health. 16, 66.
    Pubmed CrossRef
  8. Ebabe Elle, R, Gaillet, S, Vide, J, Romain, C, Lauret, C, Rugani, N, Cristol, JP, and Rouanet, JM (2013). Dietary exposure to silver nanoparticles in Sprague-Dawley rats: effects on oxidative stress and inflammation. Food Chem Toxicol. 60, 297-301.
    Pubmed CrossRef
  9. Tang, J, Xiong, L, Wang, S, Wang, J, Liu, L, Li, J, Yuan, F, and Xi, T (2009). Distribution, translocation and accumulation of silver nanoparticles in rats. J Nanosci Nanotechnol. 9, 4924-4932.
    Pubmed CrossRef
  10. Martins, ADC, Azevedo, LF, de Souza Rocha, CC, Carneiro, MFH, Venancio, VP, de Almeida, MR, Antunes, LMG, de Carvalho Hott, R, Rodrigues, JL, Ogunjimi, AT, Adeyemi, JA, and Barbosa, F (2017). Evaluation of distribution, redox parameters, and genotoxicity in Wistar rats co-exposed to silver and titanium dioxide nanoparticles. J Toxicol Environ Health A. 80, 1156-1165.
    Pubmed CrossRef
  11. Filipak Neto, F, Cardoso da Silva, L, Liebel, S, Voigt, CL, and Oliveira Ribeiro, CA (2018). Responses of human hepatoma HepG2 cells to silver nanoparticles and polycyclic aromatic hydrocarbons. Toxicol Mech Methods. 28, 69-78.
    Pubmed CrossRef
  12. Brkic Ahmed, L, Milic, M, Pongrac, IM, Marjanovic, AM, Mlinaric, H, Pavicic, I, Gajovic, S, and Vinkovic Vrcek, I (2017). Impact of surface functionalization on the uptake mechanism and toxicity effects of silver nanoparticles in HepG2 cells. Food Chem Toxicol. 107, 349-361.
    Pubmed CrossRef
  13. Li, Y, Qin, T, Ingle, T, Yan, J, He, W, Yin, JJ, and Chen, T (2017). Differential genotoxicity mechanisms of silver nanoparticles and silver ions. Arch Toxicol. 91, 509-519.
    Pubmed CrossRef
  14. Nallanthighal, S, Chan, C, Bharali, DJ, Mousa, SA, Vasquez, E, and Reliene, R (2017). Particle coatings but not silver ions mediate genotoxicity of ingested silver nanoparticles in a mouse model. Nano Impact. 5, 92-100.
    Pubmed KoreaMed CrossRef
  15. Riaz Ahmed, KB, Nagy, AM, Brown, RP, Zhang, Q, Malghan, SG, and Goering, PL (2017). Silver nanoparticles: Significance of physicochemical properties and assay interference on the interpretation of in vitro cytotoxicity studies. Toxicol In Vitro. 38, 179-192.
    Pubmed CrossRef
  16. Wang, J, Che, B, Zhang, LW, Dong, G, Luo, Q, and Xin, L (2017). Comparative genotoxicity of silver nanoparticles in human liver HepG2 and lung epithelial A549 cells. J Appl Toxicol. 37, 495-501.
    Pubmed CrossRef
  17. Guo, H, Zhang, J, Boudreau, M, Meng, J, Yin, JJ, Liu, J, and Xu, H (2016). Intravenous administration of silver nanoparticles causes organ toxicity through intracellular ROS-related loss of inter-endothelial junction. Part Fibre Toxicol. 13, 21.
    Pubmed CrossRef
  18. Jakobsen, JS, Waage, J, Rapin, N, Bisgaard, HC, Larsen, FS, and Porse, BT (2013). Temporal mapping of CEBPA and CEBPB binding during liver regeneration reveals dynamic occupancy and specific regulatory codes for homeostatic and cell cycle gene batteries. Genome Res. 23, 592-603.
    Pubmed CrossRef
  19. Luedde, T, Duderstadt, M, Streetz, KL, Tacke, F, Kubicka, S, Manns, MP, and Trautwein, C (2004). C/EBP beta isoforms LIP and LAP modulate progression of the cell cycle in the regenerating mouse liver. Hepatology. 40, 356-365.
    Pubmed CrossRef
  20. Li, Y, Ma, J, Fang, Q, and Li, X (2014). c-fos and c-jun expression in the liver of silver carp and the effect of microcystins. J Biochem Mol Toxicol. 28, 157-166.
    Pubmed CrossRef
  21. Lee, D, Lim, J, Woo, KC, and Kim, KT (2018). Piperonylic acid stimulates keratinocyte growth and survival by activating epidermal growth factor receptor (EGFR). Sci Rep. 8, 162.
    Pubmed CrossRef
  22. Brzoska, K, Meczynska-Wielgosz, S, Stepkowski, TM, and Kruszewski, M (2015). Adaptation of HepG2 cells to silver nanoparticles-induced stress is based on the pro-proliferative and anti-apoptotic changes in gene expression. Mutagenesis. 30, 431-439.
    Pubmed CrossRef
  23. Jiao, ZH, Li, M, Feng, YX, Shi, JC, Zhang, J, and Shao, B (2014). Hormesis effects of silver nanoparticles at non-cytotoxic doses to human hepatoma cells. PLoS ONE. 9, e102564.
    Pubmed CrossRef
  24. Stepkowski, TM, Brzoska, K, and Kruszewski, M (2014). Silver nanoparticles induced changes in the expression of NF-kappaB related genes are cell type specific and related to the basal activity of NF-kappaB. Toxicol In Vitro. 28, 473-478.
    Pubmed CrossRef
  25. Kawata, K, Osawa, M, and Okabe, S (2009). In vitro toxicity of silver nanoparticles at noncytotoxic doses to HepG2 human hepatoma cells. Environ Sci Technol. 43, 6046-6051.
    Pubmed CrossRef
  26. Xin, L, Wang, J, Wu, Y, Guo, S, and Tong, J (2015). Increased oxidative stress and activated heat shock proteins in human cell lines by silver nanoparticles. Hum Exp Toxicol. 34, 315-323.
    Pubmed CrossRef
  27. Garcia-Reyero, N, Kennedy, AJ, Escalon, BL, Habib, T, Laird, JG, Rawat, A, Wiseman, S, Hecker, M, Denslow, N, Steevens, JA, and Perkins, EJ (2014). Differential effects and potential adverse outcomes of ionic silver and silver nanoparticles in vivo and in vitro. Environ Sci Technol. 48, 4546-4555.
    Pubmed CrossRef
  28. Zhao, T, Zhu, Y, Morinibu, A, Kobayashi, M, Shinomiya, K, Itasaka, S, Yoshimura, M, Guo, G, Hiraoka, M, and Harada, H (2014). HIF-1-mediated metabolic reprogramming reduces ROS levels and facilitates the metastatic colonization of cancers in lungs. Sci Rep. 4, 3793.
    Pubmed CrossRef
  29. Sahu, SC, Zheng, J, Yourick, JJ, Sprando, RL, and Gao, X (2015). Toxicogenomic responses of human liver HepG2 cells to silver nanoparticles. J Appl Toxicol. 35, 1160-1168.
    Pubmed CrossRef
  30. Ahmed, MM, and Hussein, MMA (2017). Neurotoxic effects of silver nanoparticles and the protective role of rutin. Biomed Pharmacother. 90, 731-739.
    Pubmed CrossRef
  31. Felizola, SJ, Nakamura, Y, Arata, Y, Ise, K, Satoh, F, Rainey, WE, Midorikawa, S, Suzuki, S, and Sasano, H (2014). Metallothionein-3 (MT-3) in the human adrenal cortex and its disorders. Endocr Pathol. 25, 229-235.
    Pubmed CrossRef
  32. Liu, W, Worms, IAM, Herlin-Boime, N, Truffier-Boutry, D, Michaud-Soret, I, Mintz, E, Vidaud, C, and Rollin-Genetet, F (2017). Interaction of silver nanoparticles with metallothionein and ceruloplasmin: impact on metal substitution by Ag(i), corona formation and enzymatic activity. Nanoscale. 9, 6581-6594.
    Pubmed CrossRef
  33. Kim, S, Choi, JE, Choi, J, Chung, KH, Park, K, Yi, J, and Ryu, DY (2009). Oxidative stress-dependent toxicity of silver nanoparticles in human hepatoma cells. Toxicol In Vitro. 23, 1076-1084.
    Pubmed CrossRef
  34. Yan, HT, Shinka, T, Sato, Y, Yang, XJ, Chen, G, Sakamoto, K, Kinoshita, K, Aburatani, H, and Nakahori, Y (2007). Overexpression of SOX15 inhibits proliferation of NT2/D1 cells derived from a testicular embryonal cell carcinoma. Mol Cells. 24, 323-328.
    Pubmed
  35. Tamura, G, Olson, D, Miron, J, and Clark, TG (2005). Tolloid-like 1 is negatively regulated by stress and glucocorticoids. Brain Res Mol Brain Res. 142, 81-90.
    Pubmed CrossRef
  36. Reshma, VG, Syama, S, Sruthi, S, Reshma, SC, Remya, NS, and Mohanan, PV (2017). Engineered Nanoparticles with Antimicrobial Property. Curr Drug Metab. 18, 1040-1054.
    Pubmed CrossRef
  37. Mishra, AR, Zheng, J, Tang, X, and Goering, PL (2016). Silver nanoparticle-induced autophagic-lysosomal disruption and nlrp3-inflammasome activation in HepG2 cells is size-dependent. Toxicol Sci. 150, 473-487.
    Pubmed CrossRef
  38. Fuchs, Y, and Steller, H (2015). Live to die another way: modes of programmed cell death and the signals emanating from dying cells. Nat Rev Mol Cell Biol. 16, 329-344.
    Pubmed CrossRef
  39. Zielinska, E, Tukaj, C, Radomski, MW, and Inkielewicz-Stepniak, I (2016). Molecular mechanism of silver nanoparticles-induced human osteoblast cell death: protective effect of inducible nitric oxide synthase inhibitor. PLoS ONE. 11, e0164137.
    Pubmed CrossRef
  40. Miyayama, T, Fujiki, K, and Matsuoka, M (2018). Silver nanoparticles induce lysosomal-autophagic defects and decreased expression of transcription factor EB in A549 human lung adenocarcinoma cells. Toxicol In Vitro. 46, 148-154.
    CrossRef


e-submission

e-submission

This Article


Cited By Articles
  • CrossRef (0)

Funding Information

Services

Archives