Tri‐transformer Hawkes process via dot‐product attention operations with event type and temporal encoding. (3rd December 2021)

• Record Type: Journal Article
• Authors: Song, Zhi‐yan; Liu, Jian‐wei
• Is Part Of: Computational intelligence. Volume 38:Number 2(2022)
• Page Start: 690
• Page End: 712
• Publication Date: 2021-12-03
• Subjects: dot‐product attention -- encoding of event type -- encoding of temporal -- Hawkes process -- information -- transformer Hawkes process; Artificial intelligence -- Periodicals; Computational linguistics -- Periodicals; 006.3
• DOI: 10.1111/coin.12496
• Languages: English
• ISSNs: 0824-7935

Abstract

Abstract: Asynchronous event sequences widely exist in the real world, such as social networks, electronic medical records, financial data, and genome analysis. For modeling asynchronous event sequences in the continuous time domain, point process has become the underpinning. In the initial research stage, Hawkes process is widely used because it can capture the self‐triggering and mutual triggering modes between different events in a variety of point process functions. In recent years, due to the development of neural networks, deep point process (also known as neural point process) can learn models with the stronger fitting ability and reduce the dependence on prior knowledge by using the powerful capacity of neural networks. The proposal of the transformer Hawkes process (THP) has led to a huge performance improvement, so a new climax of the transformer‐based deep Hawkes process is set off. However, THP does not make full use of the event and temporal information underlying the asynchronous event sequence, meanwhile, if we simply take the event type encoding and temporal encoding as the sequence encoding, a single transformer may suffer from learning bias. In order to circumvent these problems, we propose a tri‐transformer Hawkes process model (TTHP), in which the event and temporal information are introduced to the dot‐product attention operations as auxiliary information to form different multihead attention, respectively, and are utilized to build three heterogeneous learners. A series of well‐designed experiments on synthetic and real‐world datasets validate the effectiveness of the proposed TTHP.