Freedom of Expression is a precious virtue of democracy. We protect it first of all for the citizen outside of any political movement, whereas inside we defend it by guaranteeing that everyone be enabled to contribute in a civilised way, using reasonable argumentations without others occupying their space, ridicule their proposals or inundate them with unfounded polemics. Freedom of expression must be the freedom of thought, not the freedom of manipulation.(2) Conceding space to polemics breaks the balance of a fair democratic debate. It isn't censorship to intervene against polemics, it is an act of protection of the working space (aka "safe space").(3)
It is no act of democracy to tolerate intolerance. Your Freedom of Expression ends where it confines with the rights and dignity of your neighbour and the project in common. This document speaks of the creation of mechanisms that effectively protect the right to Freedom of Expression and, even more, the dignity of the participants.
The digital ambience favours disinhibition,(4) facilitates rapid exchanges in communication and,(5)(6) in absence of non-verbal cues (expressions, voice tones, postures etc.), enhances the risk of failed comprehension and of inadequate emotive reactions.(7) The Internet has abundantly shown how online debate is much likelier to lead to misunderstandings, disputes, nonconstructive behaviours and other conflictual dynamics.(8)(9)(10)
The speed of exchanges, once the misunderstanding is established, allows for a rapid degradation of the climate of communication; even when it is just a humoristic form of interaction among long-term contendants, this can anyhow cause damage the motivation or public image of the movement in the way this is perceived by third parties assisting the spectacle: be it other members, electorate, media representatives, political allies or opponents.(11)(12) Debates, that trigger a feeling of shame towards the movement, can have catastrophic consequences for the social cohesion, the ability to grow and to act efficiently.
Consequently, we apply the same methods of maintenance of good coexistence also in physical spaces, just as they have proven necessary in digital spaces.
The Net recommends a simple principle for the resolution of problems: "Don't Feed The Troll." In democratic spaces, this approach has failed as there are always third parties listening which will be influenced by the "provocations", being the troll is thus politically effective. The Net awards the disturber and punishes the educated contributors. (13)(14)(15)(16) This is a serious problem as it distorts democracy and hurts the ability to develop political work oriented at evidence and rationality in many places around the net.
Another apparently simple way to resolve the problem would be to let the justice system of the hosting nation take care of possible issues of inner justice. The members of the participation platform would, in this case, sue political aggressors in a real world court in order to obtain a just defense of their political proposals. The Italian Pirates have tried this way in the past: there has never been any legal proceeding of this kind. What generally happens is that the educated political proponent gives up and finds themselves a less confrontational political project to contribute to, leaving the project in the hands of the more aggressive types. The project therefore loses participants.
We researched the history of digital sociology and found no alternative: somebody has to have the official role of guarantor of Freedom of Expression. It isn't possible to let everybody do it, because you cannot allow everyone to moderate or suspend participants from the participation platforms. But, if no moderation or suspension is ever executed, there will be no respect for anyone who intervenes in a litigation. This is the dramatic difference between digital and physical socialisation: a person's natural authority does not manifest itself on the Net. Whoever intervenes out of personal initiative ("civil courage") online, typically appears like an authoritarian buffoon, not like a hero or a respected person like we are used to in the physical world. Also, nobody has a political interest in making enemies and risking social damage. (17) Over time, a culture of overbearingness and informal power groups is established, leaving the political project to oscillate between the inability to act in a coherent way and the creation of de-facto authoritarian structures.(18)(19)
Assuming we have a consensus in wanting that the digital participation platform have the characteristics of a "commons", we need rules to moderate or suspend persons that are damaging the social procedures of the project, without however censoring them in a democratic sense.(20)(21)(22) It becomes necessary that someone be in charge of applying such rules and, unfortunately, for as much as we would have liked to automate such a job, it is necessary that this person be of flesh and blood. It is then among the duties of the justice body of the movement to guarantee that the moderation does not degenerate into censorship. That is how the moderator needs to be an officially delegated role. Only a person in charge has not only the authority but also the obligation to ensure that motivating peaceful coexistence takes place. If noone has such a role, over time nobody will care about it at all.
Justice must work for all and without victims having to fight for their rights. When quiet persons that would like to contribute, notice that there are bullies that insult them and noone comes to their defense, they probably won't start to justify themselves. They simply abandon the project. Frequently it isn't even possible to detect who the responsible of a fight is.(23) Yet, it has never worked to just blame both contendants, the actual offender will always gain an advantage from that.
Should a Moderator mistakenly moderate the "victim" rather than the "aggressor", they will however have contributed to the inner peace of the platform as they stopped the process of escalation of the dispute. Elementary things any parent should know: it isn't always possible to be just, but it is important to stop the excess of aggressions, especially in a project on the Internet wherein people do not perceive the reactions of others, sitting in front of a different screen in a different place. After the immediate intervention, the Moderator themselves can delve into aspects of justice, observing the situation, consulting with other moderators, or, by themselves submitting the case to the Court of Arbitration.
A sanction or a moderation should not be considered a punishment but instead a method to reintegrate the eccentric individual and re-establish a climate of coexistence and collaboration.(24)(25) It would be ideal to blow out the flames immediately after ignition and have rules be respected to become not only at a tight-knit and positive community, but also to succeed at the inclusion of a person that in their hearts has the right intentions. Therefore it is not helpful to let the other participants become aware of the interventions of moderation as they could damage the reputation of the moderated person.
The duty of the Moderators (aka "Vibes Watchers" or "Responsibles for Coexistence") is to oversee the debates and intervene, from heated misunderstandings, degeneration of tones to breaches of the Code of Conduct like ad-hominem attacks,(26)(27) or other troll-like behaviours aimed at disturbing or manipulating the discussions.(28)
The political reasonings of the movement need to be communicated to third parties and should therefore be rooted in rational argumentations and/or scientific data. A consensus of mere opinion is not acceptable with its demagogic risks, potentially manipulatible and manipulative. To insist on hasty unfounded opinions is to be considered a lack of respect towards rational and/or scientific argumentations.
Only if the problems of coexistence perpetuate, the interventions of the Moderators start having a disciplinary character. The judgement of the Moderators must however be considered preliminary in preparation of a possible examination by the Court of Arbitration, if any other member submits the case to them.(29) Ideally, the interventions are of such short impact, that the resulting civil coexistence is considered of higher value than the expectation of perfect justice.
Moderators of Coexistence bear the responsibility that possible diatribes are immediately halted without causing damages to the movement. A hesitation or disinterest in executing such duty can itself motivate disciplinary actions by the Court of Arbitration.
Moderators are aware that they may not be in the know of all the factors that led to a dispute, as in the case of a precedent among the contendants. It is not a part of their role to have knowledge of suchlike information - only the Court of Arbitration has the job of dealing with private and interpersonal information in order to ensure actual justice. Moderators cannot achieve that purpose, but they might nonetheless incidentally learn of private information. Moderators are obliged to treat such information with maximum confidentiality and only exchange it with the Court of Arbitration for the purposes of justice and with the Coordinating Group in cases of strategic relevance for the movement.(30)
Interventions are executed with gentleness and firmness, avoiding in the best ways to hurt the dignity of the involved persons, with particular care for the opinion the other participants make of them.(31)(32)(33) Transparency needs to manifest itself in the political content, not in the publication of social affairs. The Moderator interacts directly with the involved individuals without expressing any disciplinary warnings in public.(34) The interventions leave no trace except for a later examination by the Court of Arbitration, the Coordinating Group or other Moderators.
A disciplinary action on a participant can be their temporary suspension from participation. In the digital realm, a simple deceleration of exchanges in communication can frequently be sufficient to calm down emotions and augment the space for reflection. The person is given a new chance to integrate without the other participants knowing of the intervention. If the pride of the person has been respected, it is more likely that they will concentrate on contributing politically without interpersonal references. The participant must be firmly motivated to produce constructive and respectful contributions early on, otherwise the Moderators risk carrying too much workload.
From this perspective it is a healthy habit to prevent quarrels by intervening at the first manifestation of tones that may be understood in a wrong way.(35)(36) A typical intervention of the Moderator could be: "Bear patience with me, but your contribution could in my opinion be misunderstood. Please find a wording such that it cannot be interpreted as personal criticism."(37) The tools of the Moderators are ideally the following:
The moderation, that is the examination of the messages that one or more or all participants contribute to a conversation before their publication. Some of the best discussion spaces on the Internet, regarding the quality of the contributions, are entirely under moderation.(38)(39)(40)(41)(42)(43)(44)(45) Moderators must not exercise any censorship regarding the political expression however, except where the rules of the movement expect so.(46)
Suspension of participants wherever moderation isn't possible or practical. Frequently a suspension of just one day is sufficient to inhibit the process of escalation of a clash. According to Elinor Ostrom, suspensions shall be gradually prolongued in case of repeating offenses.(47)(48) The debating platform should allow the removal of detrimental content, allowing its author, during the pause of reflection, to reconsider the formulation. It is not enough that the content be hidden but still accessible as it can damage the dignity of its author or of referenced persons. Such hidden-from-view statements can also be of strategic interest for opponents of the movement.
The reorganisation of discussions into threads and suitable categories, to ensure that they will remain on-topic and that it will be easy to distinguish which debates are of constructive and operative kind compared to those that promote opinions the movement has already rejected or are otherwise in partial divergence with the principles laid out in the Code of Conduct.(49)(50)(51)(52)
Many social spaces offered by the Internet, like blogs, forums and social networks, aren't under administration of the movement and therefore do not allow for the regular operation of the Moderators. In these spaces it is not possible to hold a political debate guaranteed to be respectful of civil sociality and of the founding principles of the movement.(56)
A mere expression of a personal opinion of a member, but non officially of the movement, can give a devastating impression to entire portions of electorate. It is of strategic importance for the movement that in these spaces the members make a special effort to solely argument and communicate official positions of the Permanent Assembly. In case of damages to the movement, the Court of Arbitration must immediately execute sanctions on their authors.
Most democratic constitutions guarantee a legal right of Freedom of Expression allowing any member to express themselves in public, but damages caused by consequence of that, even if not quantifiable, can imply disciplinary consequences up to the exclusion from the political project.
Restorative Justice(57) is a method to re-establish peace between the participants by nearly public ventilation and agreement on measures to restore justice. The simplicity of this method is inviting, it doesn't however take the needs for privacy of the participants in consideration, whereby a Court of Arbitration can serve a better job. Also, it only comes into play after damage done, whereas our approach is oriented on prevention.
|||(it) The official location of the document is available to party members alongside the voting details in the permanent assembly at i6329. "pages.de" is not an official domain of the Italian Pirates, just a place where lynX has implemented his multi-lingual text mark-up tool.|
|||(de/en) The Illusory World of Facebook & Co., ZDF documentary on commercial and political manipulation on the Internet, 2016.|
|||(en) Criado-Perez, Women’s Aid Speech on Cyber-Harassment, 2013: «But this free speech I’ve discovered, the free speech of women, is under attack. And it’s under attack as much from people who tell us not to feed the trolls, to stop attention-seeking, to keep quiet and not be controversial, as it is from men who send us rape threats every time we open our mouths, or those who call us Nazis for objecting to this. Freedom of speech is a beautiful thing. But in its current incarnation it serves the interests of the powerful, rather than the powerless. Like so many other liberal concepts, when it exists in a society where substantive equality, as opposed to formal or legal equality, has yet to be achieved, where we have equal pay acts, but no equal pay, it can be as oppressive as it is liberating. And if we don’t question this simplistic understanding we have of free speech as a society, we will continue to live in a society where it’s ok that women don’t have a voice – politically, publicly, and socially.»|
|||(en) The online disinhibition effect|
|||(en) Seabrook, My First Flame (1994); «I suppose the guy could have written me a nasty letter: […] he probably wouldn’t have mailed the letter; he would have thought twice while he was addressing the envelope. But the nature of E-mail is that you don’t think twice. You write and send.»|
|||(en) The Universal Rules of Civilized Discourse, 2013: «Conversations are about frequent back and forth replies, to be sure. But every reply should make an honest effort to respond to, and build on, the conversation.»|
|||(en) Rice, Love, Electronic Emotion, 1987: “Disinhibition” may occur “because of the lack of social control that nonverbal cues provide.”|
|||(en) Reid, Electropolis, Communication and Community On Internet Relay Chat, 1991: «The lack of social context cues in computer- mediated communication obscures the boundaries that would generally separate acceptable and unacceptable forms of behaviour. Furthermore, the essential physical impression of each user that he is alone releases him from the social expectations incurred in group interaction. Computer-mediated communication is less bound by conventions than is face-to-face interaction. With little regulating feedback to govern behaviour, users behave in ways that would not generally be acceptable with people who are essentially total strangers.»|
|||(en) Kiesler, Siegel, McGuire, , Social Psychological Aspects of Computer-Mediated Communication, 1984: «People in computer-mediated groups were more uninhibited than they were in face-to-face groups as measured by uninhibited verbal behavior, defined as frequency of remarks containing swearing, insults, name calling, and hostile comments. […] First, […], we concluded that our findings are generalizable to adults and nonstudents as well as to undergraduate students. Second, from comparing experienced and inexperienced computer network users, we concluded that our results apply not just to novices but also to people who use computers often and for whom electronic mail and message systems as well as simultaneous discussion systems are familiar. Third, we also have compared strangers and friends and obtained similar results. […] How might we explain the results as a whole? There are at least three alternatives, having to do with (a) difficulties of coordination from lack of informational feedback, (b) absence of social influence cues for controlling discussion, and (c) depersonalization from lack of nonverbal involvement and absence of norms. […]»|
|||(en) Seabrook, My First Flame (1994); «Before long, the community was so absorbed in an attempt to identify the bad apple that constructive discourse ceased. The group posted many messages imploring whoever was doing this to stop, but the person didn’t, and the community was destroyed. […] Everywhere I went in the newsgroups, I found flames, and fear of flames. In the absence of rules, there is a natural tendency toward anarchy on the net anyway, and in some stretches I’d come upon sites that were in complete chaos, where people had been flaming each other non-stop, absolutely scorching everything around them, and driving all the civilized people away. Sometimes I’d arrive at a dead site long after a flame war broke out; it was like walking through what was once a forest after a wildfire.»|
|||(en) An Interactional Reconceptualization of "Flaming", Example "B" shows how insider messages can be perceived as offensive by silent third party readers.|
|||(en) Proboards Forum Moderating Tips: «Don't tolerate flaming; disagreements are fine but any more than that will discourage guests from joining and drive quality members from your forum. Make sure everyone has a chance to participate.»|
|||(en) Phillips, Don't feed the trolls? It's not that simple, 2013: «First of all, “don’t feed the trolls” frames conversations about aggressive online behaviors solely in terms of the aggressor. Even if a person avoids feeding the trolls (and/or the person accused of trolling), he or she is still playing into the aggressor’s hands.[…] If only the target hadn’t fed the trolls, the argument goes, the trolls wouldn’t have done what they did!»|
|||(en) Golden, Why “Don’t Feed the Trolls” Falls Short|
|||(en) Criado-Perez, Women’s Aid Speech on Cyber-Harassment, 2013: «If there’s one thing I want to come out of what happened to me, it’s for the phrase “don’t feed the trolls” to be scrubbed from the annals of received wisdom. Not feeding the trolls doesn’t magically scrub out the image in your head of being told you’ll be gang-raped till you die. […] Not talking about this is not going to make abuse and misogyny go away. On the contrary, it will help it to thrive. […] And the police had not helped them. The police had told them to lock their accounts, to stop tweeting controversial things»|
|||(en) Gibson, “Don't Feed the Trolls” and Other Stupid Advice, 2013|
|||(en) Seabrook, My First Flame (1994); «If this had happened to me in the street, I could have used my status as a physically large male to threaten the person, but in the on-line world my size didn’t matter.»|
|||(en) Jo Freeman, The Tyranny of Structurelessness|
|||(en) Bailey Lamon, Why this Radical Leftist is Disillusioned by Leftist Culture|
|||(en) Elinor Ostrom, Governing the Commons, 1990|
|||(en) Phillips, Don't feed the trolls? It's not that simple, 2013: «The first and most basic way to play Ruin This Asshole’s Day is to shut them down, ideally by unceremoniously deleting their comments. […] these problems aren’t easily solved, particularly when online abuse has a group dimension, or when the abuse occurs on unmoderated or poorly-moderated platforms.»|
|||(en) Singal, Most Comments Are Horrible—Sites Look for Ways to Make Them Better, 2012|
|||(en) Wikipedia:Flaming: «Resolving a flame war can be difficult, as it is often hard to determine who is really responsible for the degradation of a reasonable discussion into flame war.»|
|||lynX likes to think that the Moderator is more like a public relations department, helping each participant not to make a Trump of themselves.|
|||(en) Restorative Justice|
|||(en) The Universal Rules of Civilized Discourse, 2013: «You may wish to respond to something by disagreeing with it. That’s fine. But, remember to criticize ideas, not people. Please avoid: Name-calling. Ad hominem attacks. Responding to a post’s tone instead of its actual content. Knee-jerk contradiction. Instead, provide reasoned counter-arguments that improve the conversation.|
|||(en) Paul Graham, How to Disagree, 2008: «But though it's not anger that's driving the increase in disagreement, there's a danger that the increase in disagreement will make people angrier. Particularly online, where it's easy to say things you'd never say face to face. If we're all going to be disagreeing more, we should be careful to do it well.»|
|||(en) phpBB forum rules: «Arguing with team members after having received a warning will lead to an immediate additional warning. […] Users who feel they have been unfairly warned are welcome to contact the relevant team leader. Any attempt to circumvent a temporary ban or other moderator action will lead to a permanent ban of your account(s). Circumvention includes re-registering or using an already registered account under a non-banned username. Other examples include changing IP addresses, using a new email account or other action that can be taken to evade moderator action to hide your identity as the owner of the sanctioned account.|
|||(en) Proboards Forum Moderating Tips: «Be discreet and maintain confidentiality regarding moderator activities and member information.»|
|||(en) Twenge et al, Social Exclusion Decreases Prosocial Behavior, 2005: «The implication is that rejection temporarily interferes with emotional responses, thereby impairing the capacity for empathic understanding of others, and as a result, any inclination to help or cooperate with them is undermined.»|
|||(en) Proboards Forum Moderating Tips: «Be respectful of all members, each other, and represent your forum with professionalism. Consider member feelings before hitting the submit button, both in public forums and private.»|
|||(en) Pit Schultz, Mute in Conversation: «With the moderation: it is also a contradictory role. The less the moderator appears the better the channel flows. […] there is a whole empirical science behind it, how to bring the nettime ship through dark waters... how to compress and expand, how to follow the lines of noise/pattern instead of absence/presence...»|
|||(it) Programma del Partito Pirata Italiano: Trasparenza e Privacy|
|||(en) Vibes Watchers in Randy Schutt's Consensus Notes, Nonviolent Action Handbook, Reflection Manual, Nonviolence Training|
|||(en) Proboards Forum Moderating Tips: «Watch out for trolls and spammers. Always be vigilant and proactively seek them out and take action to minimize their annoying posts. Watch for members abusing other members.»|
|||(en) Proboards Forum Moderating Tips: «Communicate with unruly members and advise them of the consequences of disruptive behavior. Edit postings when necessary so that they conform to the forum's content guidelines.»|
|||(en) nettime.org, moderated mailing lists|
|||(en) Geert Lovink, "The Moderation Question: Nettime and the Boundaries of Mailing List Culture.", 2002, ISBN 978-0-262-62180-9|
|||(en) Internet Privacy Engineering Network (IPEN) mailing list|
|||(en) Myers, Stencel, Carvin, Comments On NPR: The Right Chord With Less Discord, 2011|
|||(en) Smith, A quick history of 4chan and the rightists who killed it, 2015|
|||(en) Edward Tufte, Moderating internet forums: What's smart, not what's new: «As clearly indicated to potential contributors, we do a lot of deletingonly about half of all submitted contributions survive for more than a month. This doubtless hurts a few feelings but substantially raises the quality of the board. Very few published contributions are edited at all, other than silently to correct spelling, update an URL, or to delete a sour note in an otherwise good answer. Our view is that every contribution to Ask E.T. should advance the analytical quality of the thread. We particularly seek to avoid the chronic internet disease of "All Opinions, All the Time." The idea is to have an interesting and excellent board on analytical design that serves the content and the readers, not a board logging every attempt at publication. We also are ruthless in deleting contributions with incivilities, rants, taunts, and personal commentary on other contributors.»|
|||(en) Blackboard Help: «As an online instructor, your role is to facilitate the conversation and exchange of ideas on the discussion board. You need to ensure that students feel comfortable to share, while also monitoring responses and keeping everyone focused and on track. At the same time, you want to be careful not to dominate or impede the flow of the discussion. Occasionally, students may introduce inappropriate material for the class discussion. Depending on the maturity and sensitivity of the students in your course, you may need to review student posts for inappropriate content before sharing posts with the rest of the class.»|
|||(en) Wisegeek, What is a moderated forum?: «Though the idea of voluntary cooperation is idealistically appealing to responsible persons worldwide and has been a great success, its strength is also its weakness. Irresponsible people can use unmoderated forums for their own purposes to harass, harangue and otherwise foil a forum’s charter, at least temporarily, by choosing to ignore netiquette. In most cases trolls like this can be ignored and they will eventually move on, but there are situations in which a moderated forum is more productive. […] Some people do not appreciate a moderated forum as the Internet has always represented the very vanguard of freedom. It is seen as a type of censorship or policing in this case. However, Internet traffic has grown tremendously since the mid-1990s, and with that enormous growth comes a proportionate percentage of those unwilling to respect netiquette.»|
|||(en) Blackboard Help: «Without vigilance on the your part, even discussions starting out with ample excitement can dwindle as the term progresses. The art of moderating involves finding the right balance between guiding the conversation and standing back to allow students to discover new ideas.»|
|||(en) Elinor Ostrom|
|||(en) phpBB forum rules: «phpBB operates a three strike policy. Users will be warned a maximum of three times for any and all offences in a three month period. If the need arises for a fourth warning a temporary ban will be put in place of between 1 to 7 days. […] An exception to the three strike rule applies when users contact team members personally (via any method) and post insulting, indecent or vulgar material. Such users may be subject to an immediate permanent ban.»|
|||(it) Regolamento del Partito Pirata contenente il Codice di Condotta|
|||(it) Categoria per contenuti non pienamente conformanti con il Codice di Condotta|
|||(en) Proboards Forum Moderating Tips: «Keep posts clean and move off-topic posts to appropriate categories. […] Delete or move single posts or entire threads when necessary. Lock or unlock posts as required.»|
|||(en) The Universal Rules of Civilized Discourse, 2013: «Make the effort to put things in the right place, so that we can spend more time discussing and less cleaning up. Don’t start a topic in the wrong category. Don’t cross-post the same thing in multiple topics. Don’t post no-content replies. Don’t divert a topic by changing it midstream. Don’t sign your posts – every post has your profile information attached to it. Rather than posting “+1” or “Agreed,” use the Like button. Rather than taking an existing topic in a radically different direction, use Reply as a New Topic.»|
|||(en) Afterlife Knowledge: A Peer Moderated Forum|
|||(en) The Universal Rules of Civilized Discourse, 2013: «As long as there have been web forums, there have been moderators – whose job ranges from cheerleader to facilitator to police to janitor. Sometimes their relationship with users can become needlessly adversarial, and the community guidelines often reflect the tension between these roles. This can make the job thankless and contributes to high turnover rates, not to mention friction within the community. Trust and reputation systems make it feasible to empower the community to take on some of the more “janitorial” moderation functions. Which is a godsend, since the problems like spamming and trolling are mentioned prominently in every set of community guidelines we reviewed! […] Moderators have special authority; they are responsible for this forum. But so are you. With your help, moderators can be community facilitators, not just janitors or police. When you see bad behavior, don’t reply. It encourages the bad behavior by acknowledging it, consumes your energy, and wastes everyone’s time. Just flag it. If enough flags accrue, action will be taken, either automatically or by moderator intervention. […] Calling someone a “troll” in a post, even if true, is prohibited.»|
|||(en) phpBB forum rules: «Members are asked to not act as “back seat moderators”. If members note an issue which contravenes something in this policy document they are welcome to bring it to the attention of a member of the Moderator Team. Please use the “post report” feature to report posts. Do not respond to such topics yourself. Members who constantly “act” as moderators may be warned.»|
|||(en) Sierra, Why the Trolls Will Always Win, 2014: «Most of the master trolls weren’t active on Twitter in 2007. Today, they, along with their friends, fans, followers, and a zoo of anonymous sock puppet accounts are. The time from troll-has-an-idea to troll-mobilizes-brutal-assault has shrunk from weeks to minutes. Twitter, for all its good, is a hate amplifier. Twitter boosts signal power with head-snapping speed and strength. Today, Twitter (and this isn’t a complaint about Twitter, it’s about what Twitter enables) is the troll’s best weapon for attacking you.»|
|||(en) Restorative Justice|