Cognitive Modelling and Learning for Multimedia Mining and Understanding

A Novel Real-Time, Lightweight Chaotic-Encryption Scheme for Next-Generation Audio-Visual Hearing Aids Abstract Next-generation audio-visual (AV) hearing aids stand as a major enabler to realize more intelligible audio. However, high data rate, low latency, low computational complexity, and privacy are some of the major bottlenecks to the successful deployment of such advanced hearing aids. To address these challenges, we propose an integration of 5G Cloud-Radio Access Network (C-RAN), Internet of Things (IoT), and strong privacy algorithms to fully benefit from the possibilities these technologies have to offer. Existing audio-only hearing aids are known to perform poorly in noisy situations where overwhelming noise is present. Current devices make the signal more audible but remain deficient in restoring intelligibility. Thus, there is a need for hearing aids that can selectively amplify the attended talker or filter out acoustic clutter. The proposed 5G IoT-enabled AV hearing-aid framework transmits the encrypted compressed AV information and receives encrypted enhanced reconstructed speech in real time to address cybersecurity attacks such as location privacy and eavesdropping. For security implementation, a real-time lightweight AV encryption is proposed, based on a piece-wise linear chaotic map (PWLSM), Chebyshev map, and a secure hash and S-Box algorithm. For speech enhancement, the received secure AV (including lip-reading) information in the cloud is used to filter noisy audio using both deep learning and analytical acoustic modelling. To offload the computational complexity and real-time optimization issues, the framework runs deep learning and big data optimization processes in the background, on the cloud. The effectiveness and security of the proposed 5G-IoT-enabled AV hearing-aid framework are extensively evaluated using widely known security metrics. Our newly reported, deep learning-driven lip-reading approach for speech enhancement is evaluated under four different dynamic real-world scenarios (cafe, street, public transport, pedestrian area) using benchmark Grid and ChiME3 corpora. Comparative critical analysis in terms of both speech enhancement and AV encryption demonstrates the potential of the envisioned technology to deliver high-quality speech reconstruction and secure mobile AV hearing aid communication. We believe our proposed 5G IoT enabled AV hearing aid framework is an effective and feasible solution and represents a step change in the development of next-generation multimodal digital hearing aids. The ongoing and future work includes more extensive evaluation and comparison with benchmark lightweight encryption algorithms and hardware prototype implementation.

An Improved Deep Polynomial Network Algorithm for Transcranial Sonography–Based Diagnosis of Parkinson’s Disease Abstract Transcranial sonography (TCS) is a valid neuroimaging tool for the diagnosis of Parkinson’s disease (PD). The TCS-based computer-aided diagnosis (CAD) has attracted increasing attention in recent years, in which feature representation and pattern classification are two critical issues. Deep polynomial network (DPN) is a newly proposed deep learning algorithm that has shown its advantage in learning effective feature representation for samples with a small size. In this work, an improved DPN algorithm with enhanced performance on both feature representation and classification is proposed. First, the empirical kernel mapping (EKM) algorithm is embedded into DPN (EKM-DPN) to improve its feature representation. Second, the network pruning strategy is utilized in the EKM-DPN (named P-EKM-DPN). It not only produces robust feature representation, but also addresses the overfitting issues for the subsequent classifiers to some extent. Lastly, the generalization ability is further enhanced by applying the Dropout approach to P-EKM-DPN (D-P-EKM-DPN). The proposed D-P-EKM-DPN algorithm has been evaluated on a TCS dataset with 153 samples. The experimental results indicate that D-P-EKM-DPN outperforms all the compared algorithms and achieves the best classification accuracy, sensitivity, and specificity of 86.95 ± 3.15%, 85.77 ± 7.87%, and 87.16 ± 6.50%, respectively. The proposed D-P-EKN-DPN algorithm has a great potential in TCS-based CAD for PD due to its excellent performance.

Modeling Marked Temporal Point Process Using Multi-relation Structure RNN Abstract Event sequences with marker and timing information are available in a wide range of domains, from machine log in automatic train supervision systems to information cascades in social networks. Given the historical event sequences, predicting what event will happen next and when it will happen can benefit many useful applications, such as maintenance service schedule for mass rapid transit trains and product advertising in social networks. Temporal point process (TPP) is one effective solution to solve the next event prediction problem due to its capability of capturing the temporal dependence among events. The recent recurrent temporal point process (RTPP) methods exploited recurrent neural network (RNN) to get rid of the parametric form assumption in the density functions of TPP. However, most existing RTPP methods focus only on the temporal dependence among events. In this work, we design a novel multi-relation structure RNN model with a hierarchical attention mechanism to capture not only the conventional temporal dependencies but also the explicit multi-relation topology dependencies. We then propose an RTPP algorithm whose density function conditioned on the event sequence embedding learned from our RNN model for cognitively predict the next event marker and time. The experiments show that our proposed MRS-RMTPP outperforms the state-of-the-art baselines in terms of both event marker prediction and event time prediction on three real-world datasets. The capability of capturing both ontology relation structure and temporal structure in the event sequences is of great importance for the next event marker and time prediction.

Perceptions or Actions? Grounding How Agents Interact Within a Software Architecture for Cognitive Robotics Abstract One of the aims of cognitive robotics is to endow robots with the ability to plan solutions for complex goals and then to enact those plans. Additionally, robots should react properly upon encountering unexpected changes in their environment that are not part of their planned course of actions. This requires a close coupling between deliberative and reactive control flows. From the perspective of robotics, this coupling generally entails a tightly integrated perceptuomotor system, which is then loosely connected to some specific form of deliberative system such as a planner. From the high-level perspective of automated planning, the emphasis is on a highly functional system that, taken to its extreme, calls perceptual and motor modules as services when required. This paper proposes to join the perceptual and acting perspectives via a unique representation where the responses of all software modules in the architecture are generalized using the same set of tokens. The proposed representation integrates symbolic and metric information. The proposed approach has been successfully tested in CLARC, a robot that performs Comprehensive Geriatric Assessments of elderly patients. The robot was favourably appraised in a survey conducted to assess its behaviour. For instance, using a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree), patients reported an average of 4.86 when asked if they felt confident during the interaction with the robot. This paper proposes a mechanism for bringing the perceptual and acting perspectives closer within a distributed robotics architecture. The idea is built on top of the blackboard model and scene graphs. The modules in our proposal communicate using a short-term memory, writing the perceptual information they need to share with other agents and accessing the information they need for determining the next goals to address.

D-WASPAS: Addressing Social Cognition in Uncertain Decision-Making with an Application to a Sustainable Project Portfolio Problem Abstract Decision-making is an interdisciplinary area that has roots in mathematics, economics, and social science. Multiple-criteria group decision-making (MCGDM) is one of the most applicable areas of decision-making. Social cognition is involved in group decision-making. Therefore, it is necessary to address how decision makers (DMs) process and apply judgments and information during the process. In recent years, many approaches have been applied to MCGDM. As an important aspect of this process, uncertainty has led to the application of fuzzy sets. However, utilizing various decision-making approaches can result in different results and confusion among DMs. Moreover, using classic fuzzy sets and expressing degrees of belonging by crisp values has proven to be inadequate for uncertain decision-making environments. This paper presents a novel MCGDM approach, double-weighted aggregated sum product assessment (D-WASPAS), under interval-valued Pythagorean fuzzy (IVPF) uncertainty. The proposed approach applies knowledge measures to address the objective weights of criteria. Then, subjective and objective weights of criteria are aggregated to create a more appropriate weight. This approach considers three decision-making methods. In the first, an IVPF-ARAS (additive ratio assessment) method is extended to rank the alternatives. In the second, an IVPF-EDAS (evaluation based on distance from average solution) method is developed to rank the alternatives. In the third, a novel IVPF-COADAP (complex adequate appraisal) method is utilized for a third ranking. To aggregate the results, two steps are carried out using the WASPAS method. First, the results of the ranking approaches are aggregated. This process starts with computing the objective weights of the ranking approaches and aggregating the outcome with the subjective weights of the approaches. Then, the WASPAS method is applied to aggregate the obtained rankings and obtain a set of rankings for each DM. The second aggregation is utilized to aggregate the results for the DMs and reach a final set of rankings. Similarly, the subjective and objective weights of the DMs are applied in the WASPAS to aggregate the results. It should be noted that since the WASPAS method is utilized twice to aggregate the results, this approach is called D-WASPAS. A case study of the application of the proposed method shows that it is applicable to many multiple-criteria analysis and decision-making processes. Moreover, the results are more reliable because various decision-making methods are taken into consideration, and it is a last-aggregation process. Double-weighted aggregated sum product assessment offers a novel decision-making framework that is applicable in real-world decision-making situations. The proposed method is based on interval-valued Pythagorean fuzzy sets (IVPFSs), which would be especially applicable to uncertain situations. Also, it would enhance calculations of the process by offering more flexibility in dealing with uncertainty. Consequently, introducing this new decision-making framework and applying extended fuzzy sets would make the proposed method more widely applicable. The last-aggregation nature of this method avoids loss of cognitive information and assigning weights to the DMs, and the different ranking methods address the social cognition that leads to the judgments expressed and the final decisions.

Overview of Hesitant Linguistic Preference Relations for Representing Cognitive Complex Information: Where We Stand and What Is Next Abstract Hesitant fuzzy linguistic preference relations (HFLPRs) can be used to represent cognitive complex information in a situation in which people hesitate among several possible linguistic terms for the preference degrees of pairwise comparisons over alternatives. HFLPRs have attracted growing attention owing to their efficiency in dealing with increasingly cognitive complex decision-making problems. Due to the emergence of various studies on HFLPRs, it is necessary to make a comprehensive overview of the theory of HFLPRs and their applications. In this paper, we first review different types of linguistic representation models, including the hesitant fuzzy linguistic term set, hesitant 2-tuple fuzzy linguistic term set, probabilistic linguistic term set, and double-hierarchy hesitant fuzzy linguistic term set. The reasons for proposing these models are discussed in detail. Then, the hesitant linguistic preference relation models associated with the aforementioned linguistic representation models are addressed one by one. An overview is then provided in terms of their consistency properties, inconsistency-repairing processes, priority vector derivation methods, consensus measures, applications, and future directions. Basically, we try to answer to two questions: where we stand and what is next? The preference relations and consistency properties are discussed in detail. The inconsistency-repairing processes for those preference relations that are not acceptably consistent are summarized. Methods to derive the priorities from the HFLPRs and their extensions are further reviewed. The consensus measures and consensus-reaching processes for group decision making with HFLPRs and their extensions are discussed. The applications of HFLPRs and their extensions in different areas are highlighted. The future research directions regarding HFLPRs are given from different perspectives. This paper provides a comprehensive overview of the development and research status of HFLPRs for representing cognitive complex information. It can help researchers to identify the frontier of cognitive complex preference relation theory in the realm of decision analysis. Since the research on HFLPRs is still at its initial stage, this review has guiding significance for the later stage of study on this topic. Furthermore, this paper can engage further research or extend the research interests of scholars.

Improving the Recall Performance of a Brain Mimetic Microcircuit Model Abstract The recall performance of a well-established canonical microcircuit model of the hippocampus, a region of the mammalian brain that acts as a short-term memory, was systematically evaluated. All model cells were simplified compartmental models with complex ion channel dynamics. In addition to excitatory cells (pyramidal cells), four types of inhibitory cells were present: axo-axonic (axonic inhibition), basket (somatic inhibition), bistratified cells (proximal dendritic inhibition) and oriens lacunosum-moleculare (distal dendritic inhibition) cells. All cells’ firing was timed to an external theta rhythm paced into the model by external reciprocally oscillating inhibitory inputs originating from the medial septum. Excitatory input to the model originated from the region CA3 of the hippocampus and provided context and timing information for retrieval of previously stored memory patterns. Model mean recall quality was tested as the number of stored memory patterns was increased against selectively modulated feedforward and feedback excitatory and inhibitory pathways. From all modulated pathways, simulations showed recall performance was best when feedforward inhibition from bistratified cells to pyramidal cell dendrites is dynamically increased as stored memory patterns is increased with or without increased pyramidal cell feedback excitation to bistratified cells. The study furthers our understanding of how memories are retrieved by a brain microcircuit. The findings provide fundamental insights into the inner workings of learning and memory in the brain, which may lead to potential strategies for treatments in memory-related disorders.

Cognitive Insights into Sentic Spaces Using Principal Paths Abstract The availability of an effective embedding to represent textual information is important in commonsense reasoning. Assessing the quality of an embedding is challenging. In most approaches, embeddings are built using statistical properties of the data that are not directly interpretable by a human user. Numerical methods can be inconsistent with respect to the target problem from a cognitive view point. This paper addresses the issue by developing a protocol for evaluating the coherence between an embedding space and a given cognitive model. The protocol uses the recently introduced notion of principal path, which can support the exploration of a high-dimensional space. The protocol provides a qualitative measure of concept distributions in a graphical format, which allows the embedding properties to be analyzed. As a consequence, the tool mitigates the black-box effect that is typical of automatic inference processes. The experimental section involves the characterization of AffectiveSpace, demonstrating that the proposed approach can be used to describe embeddings. The reference cognitive model is the hourglass model of emotions.

Rank-Based Gravitational Search Algorithm: a Novel Nature-Inspired Optimization Algorithm for Wireless Sensor Networks Clustering Abstract Recently, wireless sensor networks (WSNs) have had many real-world applications; they have thus become one of the most interesting areas of research. The network lifetime is a major challenge researched on this topic with clustering protocols being the most popular method used to deal with this problem. Determination of the cluster heads is the main issue in this method. Cognitively inspired swarm intelligence algorithms have attracted wide attention in the researh area of clustering since it can give machines the ability to self-learn and achieve better performance. This paper presents a novel nature-inspired optimization algorithm based on the gravitational search algorithm (GSA) and uses this algorithm to determine the best cluster heads. First, the authors propose a rank-based definition for mass calculation in GSA. They also introduce a fuzzy logic controller (FLC) to compute the parameter of this method automatically. Accordingly, this algorithm is user independent. Then, the proposed algorithm is used in an energy efficient clustering protocol for WSNs. The proposed search algorithm is evaluated in terms of some standard test functions. The results suggest that this method has a better performance than other state-of-the-art optimization algorithms. In addition, simulation results indicate that the proposed clustering method outperforms other popular clustering method for WSNs. The proposed method is a novel way to control the exploration and exploitation abilities of the algorithm with simplicity in implementation; therefore, it has a good performance in some real-world applications such as energy efficient clustering in WSNs.